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Executive summary 



1 The study was commissioned to investigate whether or not, and if 
so, how, use of different assessment methods makes a difference 
to learner achievement and progress in the learning and skills 
sector (LSS). 

2 It sought to compare and contrast the assessment experiences 
of learners in different LSS contexts, and to identify which 
assessment regime works best in enabling learners to progress, 
in which contexts and in which sectors. 

3 This is the first comprehensive study of assessment procedures 
and practices employed across the full range of LSS contexts - 
school sixth forms, further education colleges, workplaces and 
adult learning environments. 

4 The study identified an enduring divide between post-16 
academic and vocational tracks and the different methods 

of assessment employed in those tracks. More practical forms 
of learning and assessment are favoured in the vocational 
track, but achievements attained by these methods are still 
regarded as of lower status when compared with ‘traditional’ 
examination results. 

5 The study also identified many anomalies of structure and 
practice across post-16 awards. Some would argue that these 
reflect genuine differences in the context and purpose of the 
assessment - ie fitness for purpose issues - but others would 
argue that they raise issues of equity and fairness across the 
sector. Some, such as the different status given to key skills 
testing and to GCSE passes, in the academic and vocational 
tracks, are clearly unfair and counter-productive, and warrant 
immediate change. Key skills tests in particular are not serving 
the purposes for which they were ostensibly designed, and 
should be reformed to become uniformly applicable across the 
LSS or abolished. 

6 Overall the study found that assessment methods per se do not 
directly affect learners’ choice of award or likelihood of success, 
but the association of certain awards with methods which 
employ extensive writing (coursework, exam essays, etc) does. 
Thus for example, practical tests and/or multiple-choice tests 
are seen as acceptable - and indeed unavoidable - across most 
groups of learners in the sector, especially younger trainees, 
but extensive written work is disliked and largely avoided, except 
by A-level takers. 

7 The move in recent years towards criterion-referenced 
assessment and competence-based assessment, which has 
underpinned the move towards greater transparency of intended 
learning outcomes and the criteria by which they are judged, has 
significantly benefited learners in the LSS in terms of the numbers 
of learners retained in the system and the awards which they 
achieve. Clarity in assessment procedures, processes and criteria 
has underpinned the widespread use of coaching, practice and 
provision of formative feedback to boost individual and 
institutional achievement. Detailed tutor and assessor support, 

in the form of exam coaching and practice, drafting and redrafting 
of assignments, asking ‘leading questions’ during workplace 
observations, and identifying appropriate evidence to record in 
portfolios, is widespread throughout the sector and is effective 
in facilitating learner achievement and progression. 




8 Further development of procedures and strategies noted under 7, 
above, would be helped by rendering one-year or two-year long 
programmes more accomplishable through credit accumulation. 
Achieving formal, certifiable ‘stepping stones’ along the way of an 
NVQ Level 2 or Level 3 would probably increase retention and 
completion rates. Such credit accumulation would also render 
NVQs more directly comparable with other awards at the same 
level and possibly facilitate credit transfer across awards. 

9 Electronic online testing (e-testing) is popular and effective. Wider 
use of e-testing across programmes and via wireless technology 
in workplaces could improve completion rates, pass rates and 
speed of progression in underpinning knowledge tests and adult 
basic skills tests. E-testing is most suited to multiple-choice 
formats and wider use must also attend to issues of validity and 
appropriateness. 

10 Transparency, however, encourages instrumentalism. The clearer 
the task of how to achieve a grade or award becomes, and the 
more detailed the assistance given by tutors, supervisors and 
assessors, the more likely are candidates to succeed; but 
succeed at what? Transparency of objectives, coupled with 
extensive use of coaching and practice to help learners meet 
them, is in danger of removing the challenge of learning and 
reducing the quality and validity of outcomes achieved. We have 
identified a move from what we characterise as assessment 

of learning, through the currently popular idea of assessment 
for learning, to assessment as learning, where assessment 
procedures and practices may come completely to dominate the 
learning experience, and ‘criteria compliance’ comes to replace 
‘learning’. This is the most significant challenge confronting 
assessment in the LSS: balancing the explicitness of learning 
objectives and instructional processes against the validity and 
worthwhileness of learning outcomes. 

11 The longer-term implications of 5-10, above, are that 
inconsistencies across the sector should be reviewed and a wider 
range of assessment methods should be employed across all 
awards, with the QCA and awarding bodies allowing more 
candidate choice with respect to method of assessment. In 

a social and economic environment which supposedly privileges 
the consumer over the producer, consumer choice in assessment, 
coupled with the need for equality of consumer choice across 
the LSS, should be given more attention. A range of assessment 
methods could be made available for all awards, with the 
candidate choosing that combination of methods which most 
suits their learning style and maximises their chances of success. 
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12 The ‘wider benefits of learning’ are very apparent in the data 
but are not currently identified or recorded systematically. 
Pursuing and recordingthem more specifically could go some 
way to counteracting the narrow instrumentalism noted above, 
and the pressures of accountability which ultimately drive such 
instrumentalism. It might also, if it were developed across 

the sector, feed into the wider debate about what we wish 
to achieve through vocational education. The recording of wider 
individual competencies could be pursued fairly easily, since 
evidence is likely to exist already (eg writing CVs and job 
applications, attending interviews, helping with children’s 
homework and after-school clubs, etc) The recording of increasing 
confidence and/or social capital would be more challenging, 
but might be an interesting topic for a pilot study by an 
awarding body - identifying and accrediting the collective social 
achievements of a community involved in a Sure Start programme, 
for example, rather than just the particular achievements 
of individual candidates. 

13 The balance between complying with ‘national standards’ and 
interpreting them appropriately in situ needs to be re-examined. 
Central prescription of national standards in academic subjects 
and vocational fields has been prominent for more than 20 years. 
Some would argue that such prescription is necessary, and has 
in any case arisen from widespread involvement of practitioners 
and employers in analysing national learning needs and 
specifying appropriate outcomes and competences. But this 
has led to a narrow focus on accumulating marks or elements 

of competence, thereby restricting the quality of the learning 
experience (10, above) Moreover, definitions of standards can 
never expunge local interpretation, and the evidence from this 
study and others (eg Fuller & Unwin 2003, Stasz et al. 2004) 
is that local ‘communities of practice’ constitute the context 
in which all meaningful judgements about standards are made, 
and thus should be the level of the system at which most efforts 
at capacity building are now directed. Further improvement 
of both the numbers of successful candidates, and the quality 
of the experience and awards they receive, will be dependent on 
capacity building at local level. 
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Section 2 


Introduction 



We know a good deal about the interaction of assessment, 
teaching and learning in the compulsory school sector and the 
impact of assessment on learning. What is assessed, and how 
it is assessed, is hugely influential in determining what is taught 
and how it is taught. Likewise, with respect to learning, while 
assessment can motivate learners if they are successful, it can 
also undermine confidence and capacity to learn if they are 
unsuccessful, especially if young learners meet with early failure 
(cf. Black & Wiliam 1998, Harlen & Crick 2003, Reay & Wiliam 
1999, Torrance 1995). We also know that modes and methods 
of assessment make a difference. Using a wide range of methods 
and employing formative feedback can promote learning and 
achievement (Black & Wiliam 1998, Torrance & Pryor 1998). 
However, we know very little about how assessment impacts on 
learning in the post-compulsory sector of education and training. 

This study was commissioned to address this gap. This is the first 
research project to take a comprehensive look at assessment 
methods employed across the full range of types of provision in 
the learning and skills sector (LSS) A prior review of the literature 
on assessment in post-compulsory education noted the scarcity 
of studies and the almost ‘hermetically sealed’ nature of the 
sub-sectors or ‘tracks’ within which they were contained: 

■ the ‘traditional 16-19 college-based sector’ (most studies which 
featured some discussion of assessment related to this sector, 
but they were still few in absolute number) 

■ the adult and community education sector, including Access to 
Higher Education courses (some limited studies) 

■ the work-based vocational training sector (very few studies). 

Thus while there was some evidence of assessment practices and 
impact on achievement and progress within these tracks, there 
was no comparing or contrasting of practice across tracks, and no 
discussion of what import this overall lack of an evidence base 
might carry for the sector as a whole (see Torrance and Coultas 
2004). Other recent research reviews such as Stasz et al. (2004) 
note a similar paucity of evidence with respect to assessment 
in the LSS and very limited evidence as to what teaching and 
learning approaches might actually make a difference to 
achievement and outcomes in the sector. The current project was 
commissioned to address the need for a comprehensive overview 
of assessment procedures and practices, especially at the level 
of impact on the learner. 

In particular, the research was commissioned in the context of 
a growing concern with respect to ‘over-assessment’ in the English 
education system and the seemingly unending treadmill of formal 
assessments being taken at 7, 11, 14 and 16 years of age in the 
compulsory sector, followed by further assessment at 17 and 18 
through the introduction of Curriculum 2000 (AS, A2 and AVCE) 
and key skills testing in further education and vocational training. 
Similarly, extensive formal testing of ‘basic skills’ for adult 
returners to learning had raised concerns about the impact of 
such measurement on the very process it was intended to 
underpin - progression in learning and achievement. The study 
therefore focused on the learner experience in order to: 




■ compare and contrast assessment experiences of learners in 
different settings 

■ identify what assessment regime works best in enabling learners 
to progress in which context and in which sectors 

■ identify how learners can best be supported in engaging with the 
different demands of different assessment methods. 

Data have been gathered by conducting a series of parallel case 
studies of assessment ‘in action’ across a wide variety of LSS 
settings and by a questionnaire distributed to a larger sample 
of learners derived from the case study settings. The boundaries 
of each case were established with respect to particular 
qualifications and/or awards and the contextual and regional 
factors which influence the assessment of awards in practice, 
including awarding body procedures and processes. Thus the 
case studies were designed as ‘vertical’ investigations, exploring 
a particular qualification such as AVCE or NVQ from awarding 
body through to learner, though with the emphasis on learner 
experience. The aspiration was to collect data across a wide range 
of LSS contexts and awards in order to describe and analyse 
assessment practices across the sector. Thus the study is broad 
in scope but focused in terms of topic and depth of analysis. 

The intention is to learn lessons about assessment across 
sub-sectors and contexts, by comparing and contrasting 
experience of different awards and methods of assessment 
in different settings. 

Interviews and observations have been conducted in college, 
workplace, informal adult education and school post-16 
settings in: 

■ the north of England (Manchester, Sheffield, Leeds) 

■ North Wales and the Midlands (Deeside, Stoke, Birmingham, 
Worcester) 

■ the south west of England (Bristol, Exeter). 

These have focused on: 

■ NVQ Social Care, Motor Vehicle Engineering (MVE), Sport & 
Recreation 

■ A-level and AVCE PE, Sport & Leisure and Business Studies 

■ Access to HE 

■ adult basic skills testing 

■ informal community education and accreditation via a Sure Start 
programme. 

These subjects and occupational fields were selected to 
represent the range of likely practice and experience across 
the LSS, including newer and more traditional occupational 
sectors, vocationally-oriented educational subjects, Access 
and ‘return-to-learn’ programmes, and to be reasonably 
balanced in terms of the gender, race, age and socio-economic 
status of candidates. An extension was also specifically 
commissioned by Ufi to investigate e-assessment, especially 
via Learndirect centres. 
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In total, 237 learners/candidates have been interviewed; along 
with 95 ‘assessors’ (ie all those involved in operating and 
conducting assessment within the case studies, including the 
full range of senior awarding body staff, chief and lead verifiers, 
external verifiers, employers, supervisors, college heads of 
department, tutors, internal assessors, internal verifiers, etc) 
These resulted in the production of 320pp of draft ‘case reports’ 
from which this final report is derived. Completed questionnaire 
returns were received from 260 respondents out of 890 
distributed (34% return) Full details of the sample and methods, 
and a summary of the findings from the questionnaire data, 
are included at Appendices 1, 2, 3 & 4. 

An overview of the development of assessment in the sector is 
provided in the next section to sketch out the social and political 
context in which debates about assessment in the LSS are 
located. A full understanding of the multi-faceted nature of 
assessment in the LSS and how different practices in different 
sub-sectors relate to one another - or not, as the case may be - 
warrants a preliminary discussion of this complexity in order to 
contextualise the findings set out in Section 4. The main body of 
the report (Section 4) then draws extensively on the case reports 
to present detailed evidence of the role of assessment in the LSS. 
This allows procedures and practices to be compared and 
contrasted across settings and sub-sectors. Finally, conclusions 
and recommendations are presented in Section 5. 
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Section 3 


The politics and technology of assessment in the learning 
and skills sector 



England’s post-school education and training system has evolved 
since the mid-19th century to supply an enormously complicated 
and diverse range of provision and qualifications. These have 
resisted successive attempts at rationalisation and simplification. 
Understanding the range of assessment methods used in the 
sector and the different purposes they are intended to serve 
requires some prior discussion of the educational, social and 
political development of the system. Much of current attitude 
and practice relates to, and in some important respects derives 
from, key elements of this development, especially the enduring 
academic/vocational divide. It is hard to understand the 
development of one sub-sector without also understanding what 
traditions and practices it has been developed in contrast with, 
and often in opposition to. 

The provision and qualifications which form the focus for this 
report can perhaps best be understood as operating within four 
sub-sectors or tracks, though with some cross-over and merging 
at various points: 

■ a general education system mainly for 16-19 year olds based 
on Advanced level GCE (A-levels), derived from syllabuses and 
assessment methods that have endured since the 1950s, 
accredited by the three unitary awarding bodies (ABs: AQA, 

Edexcel and OCR) and run by schools, sixth form colleges and 
further education colleges 

■ a general vocational education system mainly for 16-19 year olds 
based on the Advanced Vocational Certificate of Education 
(AVCE), derived from General National Vocational Qualifications 
(GNVQs 1991-2000), accredited by the three unitary awarding 
bodies and similarly run by schools, sixth form colleges and 
further education colleges; this sub-sector also includes BTEC 
national diplomas which parallel AVCEs and which retain a 
distinctive identity within Edexcel 

■ a training system based on National Vocational Qualifications 
(NVQs) that has been in place since 1987, along with a large 
number of other employer-led qualifications, offered by a wide 
range of awarding bodies and providers, though with City & Guilds 
as by far the largest awarding body for NVQs, which are intended 
to be assessed in the workplace though they can often involve 
simulated work environments, including college-based provision 

■ a wide array of adult and community provision, including 
specifically-designed ‘Access to higher education’ courses, 
with certificates and qualifications that offer recognition 

of achievement from basic skills in literacy and numeracy through 
to entry to higher education, accredited by a diverse range of 
awarding bodies and offered by colleges of further and higher 
education, adult education colleges and local education 
authority providers. 




Our fieldwork suggests that while the ‘traditional post-compulsory 
16-19 sector’ might be thought of as fairly uniform because 
of the age of its client group, there are academic and vocational 
routes which involve students following A-level or AVCE 
programmes; this means that it can be better understood 
as comprising two distinct tracks. On the other hand, while 
work-based routes are intended to privilege on-the-job training and 
assessment, lack of employment opportunities and work-based 
placements and resources can mean that large elements of NVQs 
are delivered and assessed in FE colleges and/or workplace 
simulations of various kinds. This can be the case for younger 
trainees, in particular, who have not gained entry to formal 
work-based programmes such as apprenticeships. Thus in some 
key respects, work-based routes are closer to general vocational 
education than might at first appear. In turn, Access to HE 
programmes, while designed for an age group which overlaps 
with the general adult and community education sector, attend 
to core elements of both general education, with respect to 
academic qualifications, and vocational work-based routes, 
since courses are increasingly being developed to provide routes 
into professions such as nursing and teaching for mature adults, 
in response to demand within these professional areas. 

Conflicting purposes of assessment in the LSS 

Superficially, these four tracks can be described in terms of the 
age-range of the target groups and the institutional base of their 
learning and assessment activities. However, their distinct 
trajectories, progression horizons and the definitions of 
‘achievement’ that pertain within each, derive far more from the 
different purposes that assessment is intended to serve within 
and across each track, and how these purposes relate to different 
conceptions and methods of assessment. Post-16 assessment is 
intended variously to: 

■ continue to serve as a preparation and selection mechanism for 
higher education (HE) so that the expansion of HE can be 
regulated with respect to distribution of limited places in elite 
institutions 

■ concomitantly identify students who do not take A-levels, or 
who achieve lower grades at A-level (Ds and Es), as ‘less able’ or 
‘non-academic’, so overall growth in the numbers of certificates 
awarded does not ‘undermine’ received notions of academic 
standards 

■ generally, nevertheless, encourage as many learners as possible 
to carry on gaining qualifications, and in particular motivate 
learners who might not otherwise stay on in post-16 education 

■ satisfy demands from different constituencies, such as 
employers’ representatives, subject associations, etc, to include 
and test what are considered to be ‘essential’ content and skills 



LSRC research report Section 3 page 10/11 



■ prepare for progression into work and job-related NVQs 

■ ameliorate previous poor levels of achievement in numeracy and 
literacy through ‘Basic Skills’ and ‘Key Skills’ provision 

■ overcome the FE/school sixth form divide 

■ raise the status of vocational training and education and in 
particular provide a meaningful work-based route through 
apprenticeships and other initiatives. 

Often, of course, the pursuit of one purpose can only be achieved 
at the exclusion of others. 

Purpose related to conceptions of achievement and 
methods of assessment 

The different demands of and purposes for the post-compulsory 
education and training system have produced diverse methods 
and combinations of methods for formal assessment and 
certification of achievement. At the same time, the policy 
imperative of access and progression has produced various 
attempts to establish broad comparability or equivalence across 
types and levels of awards: 

■ from ‘Entry level’ (basic skills) which is generally taken to be well 
below GCSE standard and actually comprises three levels (Entry 
levels 1, 2 and 3, almost akin to Key Stages 1, 2 and 3 in the 
school National Curriculum) 

■ through Foundation and Level 1 (GCSE grades D-F) 

■ to Level 2 (GCSE grades A*-C) 

■ Level 3 (two A-levels equivalent sufficient for entry to HE) 

■ Levels 4 and 5 (HE undergraduate and postgraduate equivalent). 

Foundation Apprenticeships are located at Level 2, Advanced 
Apprenticeships at Level 3. The nomenclature of equivalence, 
however, which we have been unable to escape here, is 
always articulated in relation to academic certification - GCSE 
and A-level. By discursive articulation, therefore, academic 
achievement is established and continually re-established 
as the known, tried and tested standard to which all others 
are compared. 

In turn, detailed discussion of comparability within and between 
these particular awards and levels derives from and revolves 
around assessment practices that reflect different beliefs about 
what counts as ‘achievement’, and thereby what constitutes 
‘fair assessment’. Essentially, these comprise norm-referencing, 
criterion-referencing, and the specific vocational manifestation 
of criterion-referencing: competence-based assessment. 



1 Other recent research reviews and 
explorations of data on progression 
confirm this perception of the 
enduring low status of vocational 
routes and qualifications; cf. Savory, 
Hodgson & Spours (2003), Statz ef 
al. (2004), Little & Connor (2005). 



Norm-referencing involves awarding grades by reference to the 
achievements of others in the cohort, with a rank order being 
created, then cut-off points between grades established in 
relation to the overall distribution of marks, rather than the 
absolute level of achievement of candidates. Inevitably in such 
a system, relatively few top grades are awarded. The practice 
derives both from a belief in there being a fixed ‘pool’ of ability, 
or capacity to achieve, and from which therefore the most talented 
individuals should be identified; it also derives from educational 
opportunity being historically a scarce resource (ie access to 
grammar schools, universities, etc) so that selection was the main 
purpose of assessment. Although the latter situation no longer 
pertains in an absolute sense, the former belief persists, 
particularly in the context of debates about ‘falling standards’ 

(eg at A-level) and the perception that ‘more means worse’. 

Thus while access to FE and HE opportunities have been widened 
in general, access to elite institutions is still very competitive, 
and demands the maintenance of selective mechanisms. Our 
fieldwork demonstrates that belief in such a normal curve of 
distribution is still prevalent, particularly within A-level teaching, 
but also pervades more general discussion about opportunities 
and progression, such that vocational tracks are often seen as 
‘second best’, for the ‘less able’ 1 . In turn, these are linked to 
discussion of assessment methods and the need to preserve 
externally designed and marked final examinations as providing 
reliable, uncontaminated ‘objective’ evidence of achievement. 
None of the mainstream qualification tracks outlined above 
(including A-level) now use wholly norm-referenced procedures, 
however, so the main effect and impact is at the level of belief 
and policy-development with respect to the academic/vocational 
divide. Thus it is assumed by those influenced by this frame 
of reference that ‘real achievement’ must be demonstrated 
by gaining the highest grades, while ‘real fairness’ should be 
accomplished largely through external testing and limiting 
opportunities for coaching and feedback. Awards which utilise 
assessment methods which stray too far from this historical ‘gold 
standard’ risk being labelled as lower status from the start. 



Criterion referencing involves establishing what individual 
candidates actually know and can do, irrespective of whether 
or not others in their cohort also know and can do these things. 
Aims and objectives for a course are established, along with 
clear criteria for deciding whether or not the aims and objectives 
have been achieved. This is easier said than done, but in principle, 
if a sufficient number of the objectives have been achieved, 
all can ‘pass’ the course. The driving test is the most obvious 
public example of this sort of approach. 



In practice, most courses involve grading as different levels and 
numbers of objectives are achieved and criteria are met; this 
is the basis for criterion-referenced progression and for the 
‘accumulation’ of achievement overtime. From this perspective, 
norm-referenced selective systems lack transparency, and are 
demotivating to the majority, who will, by definition, never achieve 
the highest grades. By contrast, providing clear evidence of what 
learners should achieve and have actually achieved, is claimed 
to be motivating for learners. The use of criterion referencing 
in the UK has developed significantly over the last twenty years 
or so. This growth has been largely driven by concerns for: 
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■ transparency, and improving the technical quality of assessment 
(ie improving validity and reliability by using a wider range of 
assessment methods to measure a broader range of intended 
learning outcomes) 

■ learner motivation and engagement, and the use of clear criteria 
and feedback to communicate learning goals and promote 
achievement. 

Thus ‘achievement’ within a criterion-referenced system involves 
gathering and presenting evidence by a variety of means, while 
‘fairness’ involves transparency of process and the provision 
of multiple opportunities to achieve and improve. Moreover, 
communicating clearly what learners can do and have actually 
achieved ought to be very useful for employers and HE selectors 
alike. In practice, however, selectors rarely have the time or 
inclination to delve so deep. In this respect, criterion-referencing 
lacks the simplicity, historical warrant and sheer visceral appeal 
of norm-referencing. 

Competence-based assessment is a particularly strong form 
of criterion-referencing practised in vocational environments, 
especially those of work-based learning. Here what the learner 
(often also the employee) can do, and can be seen to do, in 
relation to the tasks required of them for competent practice, 
is paramount. It is most closely associated with NVQs. Detailed 
specifications of outcomes and assessment criteria promote 
and demand ‘mastery’ (ie the meeting of all criteria) as opposed 
to the aggregation, compensation and grading common in 
examinations (ie where assessors can offset poorer performance 
in some areas by better performance in others) Candidates 
are required to show evidence of workplace competence 
in diverse forms that are relevant to demonstrating mastery - 
eg observation by supervisors and/or external assessors, written 
testimony by colleagues or managers, written assignments, 
practical tasks, oral reports and testimony. There is a strong 
emphasis on fitness for purpose and the validity of assessment 
as opposed to reliability. Candidates can repeat assessment 
tasks until they are deemed to be competent, producing 
assessment decisions of ‘not yet competent’ (‘working 
towards...’) or ‘competent’. NVQs involve learners demonstrating 
achievement when they are ready (‘readiness’), along with 
preparation and help comprising formative guidance and 
feedback and, in some cases, repeated assessments until 
the candidate achieves the outcomes. NVQs are also intended 
to be rooted in authentic workplace contexts and assessed 
by people inside those contexts. Thus ‘achievement’ is defined 
in terms of demonstrated competence in situ, while ‘fairness’ 
involves transparency of standards, criteria and procedure, 
comparability/similarity of assessment tasks and contexts, 
and multiple opportunities to demonstrate the required 
competence(s) Additionally, the content and criteria of NVQs 
are derived from ‘functional analysis’ of workplace tasks, and 
produced with the involvement of employers. They are thus 
claimed to be directly relevant to practice without any of the 
intervening ‘constructs’ which are the focus of traditional 
measurement theory. Likewise, they are claimed to be able 
to recognise and reward achievement wherever it is manifest, 
not ‘just’ in examination settings, and thus are claimed to be 
intrinsically fairer than ‘paper-and-pencil’ tests (cf. Burke 1989, 
Jessup 1991, Wolf 1995). 



Different constructions of achievement 

As is now apparent, different perspectives on assessment 
and achievement reflect contrasting assumptions about 
ability and fairness. What counts as ‘achievement’, and as 
‘fair’ (valid and reliable) methods of assessment varies between 
qualifications. The political, social, educational and technical 
dimensions to meanings of ‘achievement’ discussed above are 
crucial for understanding tutors’ and learners’ attitudes to 
achievement, and to the respective roles of formative and 
summative assessment. In AVCEs and BTEC Nationals, for 
example, formative assessment to help students improve their 
grades is integral to the educational ethos of the qualification 
(with the exception of the award of ‘distinction’ grades, discussed 
below) Such formative assessment includes guidance on draft 
assignments and close attention to the criteria for assessment 
which students are encouraged to address in detail. In NVQs, 
candidates can repeat tasks until they demonstrate competence, 
with as much guidance as necessary. By contrast, many A-level 
teachers accustomed to the one-off pressure of a summative 
examination, see formative guidance as providing an ‘unfair’ 
advantage because they believe that assessment should reflect 
the students’ ‘uncontaminated’ ability and performance on 
a particular occasion. 

Images of ‘achievement’ are also inescapably defined by those 
of ‘failure’. In popular discussions of assessment, as can be seen 
by the annual debate over GCSE and A-level results in England, 
failure is seen as a necessary adjunct to success. Pass rates 
approaching 100% are regarded as inherently implausible, even 
though an E grade at A-level is clearly worth much less than 
an ‘A’ when it comes to future opportunities for progression. This 
problem relates back to our discussion of norm-referencing and 
the requirement of elite universities for a simple selection tool to 
facilitate entry. Interestingly enough, in all of the assessment 
regimes encompassed by this study, including A-level, failure is 
regarded as an undesirable political and educational outcome, 
by candidates, tutors and policy-makers alike. For policy-makers, 
‘failure’ signals poor teaching and wasted resources; for tutors 
and candidates it signals wasted work and impoverished life 
chances. Yet lack of failure is seen to compromise the validity 
of success and, while outright failure is now avoided as far as 
possible - by the system, teachers and learners alike - A-levels 
and AVCEs still use grades as an acceptable and fair way to 
differentiate levels of achievement. In turn, levels of enrolment, 
retention and completion have become the unambiguous 
indicators of success and failure for individuals, institutions 
and system. 

Realising ‘the tracks’ in action 

To make this discussion a little more concrete, take for example 
staff in the Business sections of two colleges involved in the 
research. They taught on either AVCE or GCE A-level courses, and 
of the sample of six, five had never taught on the other pathway 
and none wanted to. Tutors had a strong sense of identity 
and affiliation with the ethos and approach of each qualification 
and this, together with their own educational backgrounds, 
shaped their notions of ‘fairness’. 
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In college 1, of the two AVCE staff, ‘Janice’ was proud of her 
identity as a non-traditional teacher and a successful ‘adult 
returner’. She did an Access to Higher Education course, 
followed by a part-time degree in Business, and then started work 
part-time in the college. ‘Mike’ did an Advanced GNVQ in Business 
in another college in the county, followed by a full-time Business 
degree and moved into teaching from an administrative role 
in the college. Their own educational experiences were therefore 
strongly attuned to those of their students. Similarly, the AVCE 
course leader in college 2, ‘Mary’, was a ‘woman returner’ who did 
an HNC in Public Administration followed by experience in Marks 
and Spencer, and voluntary work in the probation service. 

‘Neil’, the A-level tutor in college 1, had evolved his educational 
aims and ethos over 25 years of teaching A-levels in Sociology 
and Business; he had little experience of, or interest in, teaching 
general vocational qualifications. Instead, he preferred to 
concentrate on teaching GCE A-level. He had also worked as an 
examiner for A-level exam boards and spent most of his career in 
the college. Similarly, both A-level tutors in college 2 had taught 
Business A-level in the college for over 20 years; one had taught 
BTEC National in the past but now specialises in A-level. 

Their own educational and professional backgrounds led AVCE 
and GCE tutors to hold very different views about the main aims 
of each course and its assessment regime, reflecting their own 
educational and career trajectories and their affinity with the 
demands and ethos of a particular qualification pathway. These 
led to different expectations of the ‘type’ of student suited for 
each course. 

Both colleges had decided to go back to BTEC National in the 
future for their AVCE-track Business award. The decision was 
taken to privilege practical work, work placements and fieldtrips: 
‘to do all the things these kids love ... and to move away from 
all this written assessment’. Mary’s view was that this more 
specifically vocational qualification reflects ‘the way that 
students prefer to learn...’; vocational students ‘are often less 
secure and enjoy being part of one group with a small team of 
staff ... it’s more supported, it’s to do with comfort zones - 
a more protected environment’. 2 

For AVCE tutors in both colleges there was a correspondence 
between approaches to assessment and tutors’ expectations 
about students’ motivation and ability. However Neil, the GCE 
tutor in college 1, chose an exam-based A-level because it offers 
a very different view of ‘fairness’. In part, his view reflected the 
assumptions of norm-referencing, coupled with the idea that 
students should ‘take a chance’ on their own, in competition with 
others. It also reflected a belief in meritocracy and the idea that 
students should maintain a reasonable level of independent 
performance over the two years of a full A-level. For Neil, fairness 
and achievement came from assessment under exam conditions 
and without any direct help from the tutor: 

Where students perform and produce material they understand, 
that’s clear and not polluted by me. I don 't mark the scripts and 
that’s how it should be: there’s no preferential treatment, nowhere 
for the students to run and hide. In other courses, the coursework 
can be done by other people, it happens ... a student can get 
As on coursework and Ds on examined units. . . 



2 Savory, Hodgson & Spours (2003) 
also report a move back to BTEC by 
colleges dissatisfied with what they 
see as the overly academic nature of 
AVCE, as compared with GNVQ. 



Thus assumptions about the needs and preferences of different 
‘types’ of students coupled with deep-seated views about the 
efficacy of different assessment methods led tutors from the 
two qualifications to hold different views about what ‘being fair 
to the students’ meant in terms of assessment demands and 
judgments. Fairness to students was also mediated by different 
allegiances: the AVCE tutors seemed to have strong allegiances 
to their students whereas the A-level tutors saw their allegiances 
rooted in a professionalism representing their subject area on 
behalf of the awarding body. A-level tutors did not seem to 
experience this as a conflict of interest; rather, they reconciled 
their goals for students with allegiance to their subject via the 
awarding body. Having said this, however, it is also important 
to note that coaching and practising for A-level examinations 
is common, and is reported in more detail in the main body 
of the report. 

The dilemma of ‘distinction’ 

Sometimes, however, the differences between categories of 
learners and categories of assessment methods are not so clear 
cut, and perceptions of ‘fairness’ are imported from one 
assessment regime into another. In AVCE a particular challenge 
to notions of fairness comes when ideas about the provision 
of formative guidance and support come into collision with those 
of the need for objectivity and tutor disinterest. This occurs 
when students wish to achieve a ‘distinction’ in their AVCE work. 
Support is the norm, but ‘If they are looking for a distinction, 
they have to complete it as all their own work’. Here the AB 
specifications, reflecting QCA guidelines, insist that ‘distinctions’ 
are awarded on the basis of independent work. The idea of learner 
independence, and achievement accomplished without support, 
is being invoked in a different setting from that of A-level. Thus 
the idea that an assessment event was successfully completed 
independently, without much help, remains widely regarded as 
a sign of its status and validity as an achievement. This not only 
cuts across the basic assumption of transparency coupled with 
formative feedback, which energises the sector while also 
presenting both tutors and students with a very direct dilemma. 
Staff must make a judgement about a student’s likelihood of 
completing a task to the highest standard without much help, 
then leave the student to make the attempt. If the emergent or 
resultant work is judged to be of a ‘distinction’ standard (as 
opposed to ‘pass’ or ‘merit’), their original estimation would 
appear to have been justified. If, however, the work is anything 
below this standard, the subsequent feedback to the student 
may, by definition, rule out a ‘distinction’ grade, in complete 
contradiction to current policy interest in promoting ‘assessment 
for learning’. In turn, students will have to be very circumspect in 
asking for support if they wish to achieve ‘distinctions’. 



3 Savory, Hodgson and Spours (2003 
ppl5-16) make a similar observation 
about some AVCEs which they argue 
have become too academic when 
compared with their precursor, GNVQ, 
while still retaining many elements of 
the ‘mastery’ orientation of NVQ in 
the structure of their assessment. 
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Failing one element means failing all 

A further example of the way in which a clash of assumptions 
about which assessment regime is actually operational can carry 
negative consequences for candidates is found in one particular 
‘Progression Award’ in Sport and Recreation. ‘Progression Awards’ 
are designed to provide the underpinning knowledge for NVQs, 
and are taught as courses in FE colleges and other training 
providers. They are essentially college-based technical 
certificates. They should be taken in tandem with work-based 
NVQs but often are taken as stand-alone qualifications if 
candidates are not in employment. Thus progression awards, 
certainly in Sport and Recreation, can be pursued from NVQ Level 
2, to Level 3, and on to Foundation degree level, almost as a 
series of vocationally-related academic qualifications, rather than 
strictly vocational qualifications. This confusion of purpose is 
paralleled by confusion of assessment requirements, such that at 
Level 3, there is a massive assessment load, with candidates 
having to pass everything. At Level 3, there is a requirement for 
learners to complete seven course units, of which three are 
compulsory and assessed via external exams and coursework. 
The remaining four units comprise options that are assessed 
through written assignments and a series of internal tests that 
are externally verified. In addition, for each of the seven units, 
there is a requirement to complete three pieces of coursework, 
making a total of 21 pieces of written work and seven exams. 
Moreover, failure in one element means failure overall: 

If you fail one of the exams we’re having to fail the whole course, 
regardless of the all the work we've done already and the grades 
we ’ve been given; that doesn 't make any sense to me, I don 't 
believe you should fail just for one assignment or one exam. . . 

Level 3 learner 

All it needs is a student just to fail one of those exams ... last year 
there were four or five that failed one, they’re classed as failed the 
course. I think that’s wrong ... when you do an ‘A’ level which 
is say six units - you get one grade overall - that would be better. 
college curriculum manager 

Thus a vocational ‘subject’ is being assessed according to 
vocational mastery criteria (to achieve each unit the learner 
must be ‘competent’ and hence pass all assessments), 
yet these are organised in a structure that denies the flexibility 
(‘competent’/'notyet competent’) usually associated with 
competence-based assessment. In fact, certificates of unit credit 
can be issued, listing the individual units in which a candidate has 
been successful, but candidates are not eligible for a full 
certificate if they fail a unit and, clearly, this is perceived at local 
level as ‘failing’ the course. Furthermore, since the award does 
not allow for compensation or averaging across an assembly of 
coursework assignments and unit tests (as does A-level for 
example), it appears to represent the ‘worst of both worlds’ with 
respect to the merging of academic and vocational approaches 
to assessment. 3 



This confusion of purpose and structure, in assessment 
regimes which emerge from different academic and vocational 
traditions, may be one of the reasons why introducing a uniform 
unit-based and credit-based 14-19 system has received such 
widespread support in the sector (viz the Tomlinson Report) and 
is being proposed in the Qualification and Curriculum Authority’s 
‘Framework for Achievement’ (see: 
http://www.qca.org.uk/10710.html). 



Vertical ‘communities of practice’ and the realisation of 
assessment in action 

These articulations and confusions seem to derive from the 
intersection of the personal learning (and assessment) 
biographies of tutors and assessors with institutional cultures 
and regulations; the person interprets and mediates the 
regulation at the same time as they enact it. 

All qualifications in the UK are designed under guidance and 
specifications issued by the Qualifications and Curriculum 
Authority (QCA), a non-departmentai government agency which 
regulates the qualifications industry. It is intended to guarantee 
broad similarity and equivalence across awards, at the level of 
structure and procedure, laying down for example, the proportion 
of formal external testing that a qualification must include 
alongside whatever coursework, practical work, and so forth is 
allowed. The broad guidelines are then interpreted and realised 
in specific awards by awarding body teams of senior examiners 
(or product managers, as they are sometimes known in ABs), 
assessment leaders, subject and other specialists. Those who 
administrate broadly academic qualifications have to attend to 
content and criteria laid down by QCA for subjects. Those involved 
in broadly vocational qualifications have to attend to content and 
criteria laid down by standard-setting bodies such as the recently 
created Sector Skills Councils. Even at this point, and often within 
the same awarding body, the differing cultures of academic and 
vocational tracks and qualifications are embodied in different 
working teams operating with little or no contact between them. 
Award specifications and syllabuses are produced, along with 
guidance for schools and colleges to follow, which in turn are 
adopted and mapped onto the differing academic and vocational 
cultures at local level. 



4 Interestingly, a recent review 
of assessment developments in the 
post-compulsory sector for the EU 
suggests that European priorities 
for assessing ‘non-formal’ learning 
may not be driven by ‘bottom-up’ 
demands from learners, but rather 
is emerging as a ‘supply-driven 
development’ with awarding bodies 
interested in expanding their 
‘industry’ in order to argue for 
new funding from the EU, as well 
as pursue market share within the 
industry (Bjornvold 2000, p22). 



Thus three intersecting forms or levels of activity are brought to 
bear on, and produce, the reality of assessment in action: 

what we might term assessment theory and methodology - ie the 
logic and rationale which underpins and informs issues of validity, 
reliability and the impact of assessment on learning 

QCA and awarding body regulations, responsibilities and 
procedures, which have to take account of political pressures and 
policy directives as well as ‘assessment theory’, and which lead to 
the design of actual awards; it is also important to note that ABs 
are commercial organisations competing for ‘market share’ in the 
‘assessment industry’, and this will similarly influence the details 
of their procedures and the awards they offer 4 

the realisation of these awards in practice in colleges and 
workplaces, through the activities of candidates, tutors, 
supervisors and assessors. 
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In some respects, this third level could also be subdivided 
between overall college and workplace cultures of expectation 
about what assessment is and what assessors should do; and 
the particular activities of individual tutors and small groups of 
departmental or workplace colleagues which may attend to even 
more specific understandings of what assessment is ‘all about’ 
and what they are ‘allowed’ to do. 

Assessment regimes: reification and participation 

Wenger’s (1998) notion of reification and participation may be 
helpful in understanding the interactions between these forms or 
levels. These terms capture the duality with which practitioners at 
every level are faced - both ‘receiving instructions from on high’, 
while at the same time realising such instructions in action and 
hence, at one and the same time, producing their effects. Thus 
assessment might be thought of as what students, tutors and 
institutions have to do in order to meet a set of requirements that 
‘come from the awarding body’. These reflect the policy decisions 
of government agencies coupled with the educational and 
commercial values of awarding bodies. These requirements are 
usually written down (reification) But assessment can also be 
discussed in terms of experience and activity: the day-to-day 
practices of assessment and their management (participation) 

Our argument is that assessment is always about both of these 
things, at the same time. 

Reification and participation as realised in ‘communities of 
practice’ conceptualise how meaning is produced in everyday life, 
and they are usually completely interwoven. Wenger cites the 
American Constitution as an example: 

The reification of a Constitution is just a form; it is not equivalent 
to a citizenry. Yet it is empty without the participation of the 
citizens involved. Conversely, the production of such a reification is 
crucial to the kind of negotiation that is necessary for them to act 
as citizens and to bring together the multiple perspectives, 
interests, and interpretations that participation entails. 

Wenger 1998, p62 

He goes on to show how a whole range of social practices and 
meanings can be viewed as reliant on similar relationships. Of 
course, we should also note that such processes are never 
entirely benign or without competing interests being at stake. 
Individual actors can quite self-consciously attempt to circumvent 
or even subvert policy, as well as simply interpret or misinterpret 
it. Likewise, particular groups will lobby and compete to protect 
their interests and to secure the primacy of their interpretation. 
Each successive settlement (ie periodically emergent set of 
assumptions) is as much a result of struggle as of ‘negotiation’. 



Thus it is helpful to see the methods, tasks, arrangements and all 
practicalities of assessment as both what is written in ‘the rules’ 
(reification) and what people actually do in particular situations 
(participation) - and to appreciate how these are mutually 
dependent. Gathering and interpreting data from across these 
‘vertical communities of practice’, from awarding body through 
to learner and assessor, and back again, has underlined the 
necessity for keeping in view such interdependencies. While some 
tutors comment unfavourably on the constraining demands of an 
awarding body, AB representatives themselves often say similar 
things about QCA, while being dependent on tutors for their 
grounded judgments and indeed their ongoing custom. These 
kinds of dependencies are often misrecognised and presented 
only as top-down, one-way relationships, when they are much 
more dynamic and interactive than this. The particular practices 
which emerge at particular times as a result of such interaction 
can also be highly contested, however, as the current debate over 
A-level results demonstrates. While increasing the number of 
‘qualified’ people in the workforce is a goal of policy, specifically 
manifested in this example by increasing the number of A-level 
passes, the reverberations impact on other interests in the social 
system as a whole. The specific procedures and processes of any 
individual award emerge out of the informal taken-for-granted 
assumptions and mediations that individual actors invoke and 
engage in at each level of activity. However, major interests 
continue to exert considerable influence, especially with regard 
to protecting access to elite universities and securing 
the commercial viability of each of the awarding bodies. 
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Section 4 


Key themes emerging from the research 



The structure of the report 

The aims of the study were to explore learner experience of 
assessment in the LSS and in particular to try to: 

■ compare and contrast assessment experiences of learners in 
different settings 

■ identify what assessment regimes work best in enabling learners 
to progress in which context and in which sectors 

■ identify how learners can best be supported in engaging with 
the different demands of different assessment methods. 

In the course of such an investigation a large number of issues 
and problems, as well as interesting and successful practice, 
were identified. The main body of the report, which follows, 
records and analyses these issues in some detail. They comprise 
the following: 

4.1 Definitions of achievement 

4.2 Definitions of progress 

4.3 Assessment methods as used and experienced 

4.4 Supporting candidates: coaching, practising and eliciting 
evidence 

4.5 Facilitating and inhibiting learner success 

4.6 E-assessment: online testing and portfolio completion 

4.7 Key skills tests and basic skills tests 

4.8 The performance - evidence - competence continuum 

4.9 ‘Lost in translation?’ Interpreting the language of assessment 

4.10 Local communities of practice: the interpretive, mediating local 
culture(s) of education, training, employment and assessment 

4.11 ‘Innovation without change?’ The enduring academic/ 
vocational divide 

The report then finishes with an overall summary of the findings, 
including conclusions and recommendations in Section 5. 





The project has sought to investigate ‘achievement’ and 
‘progress’ in the learning and skills sector. How these terms are 
interpreted by participants will carry significant import for the 
ways in which support is construed and might be developed. And 
indeed, much data pertain to definitions of and assumptions 
about these terms, and how they are interpreted and 
operationalised in different settings. 

Achievement is fairly narrowly and instrumentally defined. 

It is routinely interpreted as securing the evidence to complete 
a portfolio and/or the ‘necessary’ or ‘expected’ grades to 
accomplish an award and proceed to further study; these are 
not necessarily the highest grades available: 

I was happy with a merit standard. 

AVCE student 

Achievement would mean getting a C or above; I’d be happy to get 
a C but I know I could do better. 

AVCE student 



Similarly, this orientation is linked to securing the 
certificate/qualification; it is not necessarily linked to the 
accomplishment of any additional or even actual practical 
competence(s), even though these may well have been acquired: 

Interviewer What would happen if you didn’t get it, if you were to drop out, 
would you lose your job? 



MVE trainee 
Int 

MVE trainee 
Int 

MVE trainee 



No, he said there’s a job for me whatever. 

So how important is it to you that you do your NVQ? 
Very important. 

But if he 's not bothered why is it important to you? 
Just having the qualifications, isn’t it. 



I’ve got to get my papers ... sol can walk into any job in the future. 
MVE trainee 



I’ve got quite a lot of experience as a mechanic because I’ve 
worked in a garage for a while but I Ve just got no papers so ... 
if anything goes wrong if I’ve got qualifications I can say well I’ve 
been in college and I’ve done that. 

MVE trainee 



Within a working environment, other rewards and incentives are 
also offered to trainees. One of the apprentices encountered 
received a £100 bonus from his employer for being ‘apprentice 
of the year’. 

Some learners comment on a wider range of achievements, 
especially with respect to personal development, or observe that 
they are ‘getting better’ at certain tasks when asked about how 
they know they are doing well, but the main focus of supervisors, 
managers, tutors and learners is on attaining grades/evidence: 

From my perspective, for me ... I want to do it, because i feel the 
need to prove I can do it, but also I had to [for my job] ... I've got to 
get it done. 

Social Care NVQ candidate 
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I’ve had four years’ experience now since being with Sure Start. 

I can do it - / know what I’m doing, you know I go in there every day 
and I am good at my job . . . but ...I just want all of that on paper . . . 
you know I want me bit of paper now to prove that I can do it ... 

I’m ... capable of doing my job. 

Adult learner involved in Sure Start programme accredited by OCN 

It’s amazing how many people really do want that bit of paper. 
adult basic skills curriculum manager 

As such, the correspondence between programme objectives, 
assessment criteria, and desired vocational outcomes, is 
crucial to genuine workforce development. If participants are 
going to pursue grades and awards, those grades and awards 
must be valid, and authentic representations of intended 
learning outcomes. 

Judgements about grades and their implications for progression 
still derive from the academic/vocational divide, with different 
tracks or pathways being offered and ‘chosen’ in the light of prior 
different (ie good/poor) GCSE and AS/AVCE results. These ‘tracks’ 
are embodied in the expectations and working cultures of both 
individual tutors and institutional organisations. In turn, kudos 
and status are also related to academic/vocational tracks; with 
vocational education and training regarded as ‘second best’, 
especially at 16-19, even if it is increasingly seen as also 
providing a ‘second chance’: 

Those who don’t do well at school can take another chance. 

AVCE Sp&L tutor 

One lad you met ... he did not do very well in his GCSEs. Courses 
like this will give him another chance. BTEC, NVQs, AVCEs are 
vocational courses, backed up with some paperwork. In the past, 
before these courses, some students would have just left school, 
some would have stayed on and failed. 

AVCE Sp&L tutor 

/ Ve got good contacts with all the schools round here. They know 
that I take apprentices on and if they’ve got somebody coming up 
that they think will be good . . . not many of them have done very 
well academically in school. A lot of them just didn’t want to be in 
school but as soon as they come here they’re doing what they want 
to do and they shine. 

MVE employer 

At the same time, however, many of the major employers of MVE 
apprentices were at pains to point out that modern cars largely 
run on programmed electronic systems of various sorts and that 
ideally they were now looking for potential technicians with good 
GCSE grades in maths and science: 

It’s all computers. You need a computer these days for diagnostic 
work. The first thing they do is plug the computer in. It’s the only 
way they can find things on modern cars. 

MVE assessor 



In general, however, the separation of ‘academic’ and ‘vocational’ 
tracks seemed fairly firm. As we have seen, staff in the Business 
sections of two colleges involved in the fieldwork taught either on 
AVCE or GCE A-level. The fieldwork reveals tutors’ strong sense 
of identity and affiliation with the ethos and approach of each 
qualification, deriving in key respects from their own educational 
backgrounds and, as one of the AVCE tutors observed, ‘We get 
the students who haven’t really got quite the grades to do the 
A-level... There’s also pressure for numbers on the A-level so the 
ones with good grades get poached from us’. On the other hand, 
A-level tutors saw a real qualitative difference in the demands 
being made on their students: ‘I just don’t see the quality in BTEC 
stuff ... a decent A-level is a qualitatively different beast in 
Business; it’s better for bright students’. 

Similarly worthy of note in this respect is that one of the other 
FE colleges involved in the fieldwork is re-branding some of its 
academic and general vocational provision as a ‘sixth form’, 
presumably to make it more attractive in competition with the 
school-based provision in the area. This is an important cultural 
issue - it seems that for many people further education college 
denotes ‘vocational’ and second best, while sixth form (and sixth 
form college) denotes ‘academic study’, leading to higher 
education. 

In different contexts, even when following vocational programmes 
themselves, respondents constantly articulate the hierarchy of 
A-levels being preferred to AVCEs, BTEC and/or Access courses. 
Thus A-levels are: 

Higher status, more well known. 

AVCE student 

They say it’s an equivalent to A-level, but of course its going to be 
viewed I suppose as lesser than an A-level. 

AVCE student 

[Employers] would go for A-level more... They hear BTEC and NVQ 
and don ’ t really know what it is. . . A-level sounds more intelligent. . . 
I just think employers need to be notified of what these 
qualifications mean . . . it’s basically discrimination, isn 't it? I ’d say 
we probably do the same amount as people who do A-level . . . but 
those students would probably not have the skills. 

AVCE student 

There was also evidence of learners preferring GCSEs to basic 
skills tests and Access Course achievements - ie many learners 
in the study are not doing A-levels, or GCSEs, but in terms of 
kudos, wish they were. 

I’d like to get a few GCSEs hopefully ... but if you've not got the 
confidence to do it because you've been suffering with dyslexia all 
your life, it takes a long time to think, ‘Yes ... I’ll go for that. ’ 
basic skills learner 

What bothers me about it more than anything is that you’re 
working towards A-level but actually at the end of it you don 'tget 
an A-level qualification. You’ve done all this work over two years, 
working to A-level standard and yet all you get is an Access 
certificate. 

Access student 
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Some students following both A-level and AVCE courses also 
noted the apparently more interesting and potentially more 
challenging nature of group work and project work in AVCEs, 
and the collaborative nature of much of the pedagogy, but 
nevertheless were still persisting with those A-levels that 
they had chosen - they were less interesting perhaps, but of 
higher status: 

I don’t say that’s how it is, but that’s how it’s seen... People have 
heard of A-level, fewer have heard of GNVQ. . . 

AVCE student 

There’s no social interaction in Economics; the teacher never 
does group work but in AVCE they do, so you might get some 
feedback . . . but cos you don 't understand it anyway, it doesn 't 
make sense and so we all go ‘we ’ll be lucky if we get an E in this. . . ' 
A-level and AVCE student 

Well, Film Studies does lend itself to group interaction and he does 
try to include the whole group, like, analysing a film . . . but they 
don’t test you until about a month before the exam and you 
practise an answer and go through it in class and that’s it ... in AS, 
you can feel alienated, like it’s only you doing it wrong, in AVCE ... 
you’re all in it together. 

A-level and AVCE student 



5 Stasz et a/. (2004) make similar 
observations, but their report also 
stands as evidence of research and 
policy interest in developing such a 
vision. As the current study was being 
completed, and the final report 
drafted, the Tomlinson report, one 
such possible vision, was being 
rejected by government as a way of 
restructuring 14-19 learning and 
achievement opportunities. Some 
elements of it may yet be 
resurrected, however. 



Overall, it would appear that we still have no positive national 
‘vision’ of what good quality general vocational education should 
look like, and therefore no coherent vision of what achievement 
should look like in such a context . 5 

Straightforward retention and completion is also regarded as 
achievement - by individuals and institutions alike - with 
retention and pass rates affecting institutional finance. Linked to 
this, there is now virtually no such thing as failure if ‘failure’ is 
interpreted in terms of completing a course or programme but not 
achieving sufficiently well to receive an award/certificate. Rather, 
failure is now defined as non-completion (of a portfolio, or of 
sufficient credits in a course of study) or not securing necessary 
grades (eg for progression to a particular HE course). 



They don’t fail, they can be withdrawn. For example, if an assessor 
observes bad practice or if somebody doesn’t seem competent 
in their work, then that would be reported to the internal verifier, 
who would speak to the line manager . . . they are mentored within 
the workplace until such time [as] their manager feels that they are 
ready to continue. That would delay their progress by six months 
or so. 

lead verifier - Social Care NVQ 



Essentially there’s no fail as such at NVQ - they’re referred. 
Obviously if they’re referred they get another opportunity to update 
their portfolio to get it to a pass standard or to demonstrate that 
they’ve met the units of competence. 
external verifier, Sport & Recreation NVQ 



Some people, through one circumstance or another, don’t attend 
enough, don’t hand in the work, don’t achieve enough credits. 
They need to achieve the minimum number of credits to get the . . . 
certificate. Without that, they can, if you like, fail. 

Access tutor 



You can fail if you just don’t do the work on time, or you don’t take 
any notice of the teachers’ advice ... [but] you can drop a couple of 
the option units if your marks aren’t any good and you can still 
pass the course. 

AVCE student 

Thus it is now most unusual for candidates to ‘fail’ A-level or 
AVCE, rather they either withdraw or achieve lower grades than 
6 Recent figures indicate avce pass they expect and hope for. 6 Moreover, those programmes which 

rates are around 80% (83% in 2002) have apparently high failure rates, such as Advanced (Level 3) 

candidate's tend to geTfewer higher Apprenticeships (AAs), often involve the candidates completing 

grades, however (see savory, their NVQs, but not bothering to take the key skills test element 

Hodgson and Spours 2003). 0 f /\/\ > anc j thus technically not completing the AA, even though 

they have become competent workers and employees - see 
Section 4.7 below on key skills tests and basic skills tests. 

The one clear exception to this ‘expulsion of failure’ identified in 
the data were ‘Progression Awards’ in Sport and Recreation, 
which, as discussed in Section 3, include a combination of course 
assignments and end-tests, with failure in any one test bringing 
failure with respect to the full certificate. There seems to be a 
specific issue here with regard to the objectives of college-based 
training, and whether or not aggregation and compensation 
across units might not be appropriate - see also Section 4.8 
below on issues of validity and reliability. 

Personal achievement 

There is some evidence in the data of the importance of 
a self-referenced, intrinsic student sense of achievement - the 
personal achievement of having ‘done something difficult’ and 
perhaps even surprised oneself. This is especially the case in the 
adult and community education sector, where such achievement 
is also linked to developing self-confidence and overcoming 
previous fears and failures deriving from school experience. 

When you ’ve gone through life not being able to read and write . . . 
personal achievement means more than a bit of paper... Actually 
being able to do it and achieve something for myself is more 
important. 
basic skills learner 

My wife 's been doing all my reading but I can 't depend on her 
all the time. . . And I can ’t keep going to my daughters or my sons 
so - bit of respect I think. .. I’ve managed with my work and that, 
but I can 't depend on my wife all the time because you never know 
what’s going to happen do you? 
basic skills learner 

Pride in what Ido... doing my best, even in horrible pieces 
of work I 've managed to get a B-t was targeted an E and I got a B. 
I’m going to try for an A. If you’d seen me at school, you wouldn’t 
believe it was the same person. 

AVCE student, now in a college 
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This sense of achievement often derives from the ‘second 
chance’ element of post-compulsory education, but seems 
to be articulated in ‘affective’ terms, relating to an overall sense 
of achievement, identity change and social progress rather than 
directly related to acquisition of skills or competences perse. 
However, the two are linked. Thus Adult basic skills (ABS) tutors 
and learners often construe achievement in terms of related 
skills, such as preparing a CV and/or applying (successfully) for 
jobs, rather than just passing ABS literacy and numeracy tests per 
se. Similarly, becoming more prepared to read to one’s children, 
and/or talk to one’s children’s teachers, and/or approach one’s 
GP about a health issue, do not just demonstrate enhanced 
confidence; they are also important ‘competences’ which are 
entirely absent from formal recordings of achievements. 

I feel a lot more confident... I’ve gotten a lot out of the [Sure Start] 
course, in order to deal with Robert’s [child’s] behaviour a lot 
more, than I actually thought I would get... We have a buddy 
system anyway, but I can ring any of them up and they talk it out 
with me... 

Sure Start OCN candidate 

I help my son at home with the homework as well, it is necessary 
for me to know English. And I need to answer him, to know the 
answer will be right, not wrong. . . I don ’t care about the exam or 
tests ...I just like learning. . . 
basic skills learner 

Some younger vocational trainees (eg MVE apprentices) identify 
a general sense of ‘doing the job better’ when asked about 
achievement, apparently relieved that they can ‘hold their own’ 
in the workplace and not be embarrassed by their novice status. 
However, they rarely go into specific detail - see Section 4.4 below 
for some examples. Older basic skills learners can sometimes be 
more specific, however, especially when comparing their 
experience to previous school failure: 

I’ve been really surprised with this English course. They’re 
explaining why things are spelled that way. I was dead excited and 
it sounds so stupid, but I was dead excited because I found out 
why hopping and hoping - one was with a p and one was pp. . . 

She 's explaining why things are spelt that way, and . . . it’s like once 
you know why and there 's a reason to it, I can do it. . . They explain 
it more and speak to you like an adult. 
student, pre-Access 

I have learnt some new stuff. I didn 't know much about the names 
of shapes and things like that, and dimensions of shapes ... don't 
know how it would help me with administration, but I think the 
fractions maybe will help me a bit in my administration job. 
basic skills learner 

I know how to write a letter ... for business, then I have improved 
my typing skills as well ...now, when I Ve finished my personal 
statement, I’ll do it on computer. 
basic skills learner 



Vocational candidates (eg Social Care) are sometimes 
encouraged to think of portfolio completion and verification as 
gaining recognition for ‘what they can do anyway’ - ie, a long- 
overdue recognition of their skills and competences by an 
accreditation system that has finally caught up with the reality of 
‘learning on the job’. Many would argue that this is an important 
element of the NVQ system, but it begs questions about the 
contribution that assessment systems should make to raising 
skill levels. Reviewing skills and accrediting individuals also 
attends to issues of safe practice and licences to practice, so it is 
clearly not an inappropriate aim for a qualifications system. But 
this also underscores the overall finding of achievement being 
interpreted in terms of accreditation and certification - in the 
case of Social Care, competence recognition, rather than 
competence development. 

It’s recognising what Ido... it’s a valuable job but now 
somebody’s recognising that. 

Social Care NVQ candidate 

They are frightened that they can 't do it because a lot ... left 
school with no qualifications and have not studied anything since 
leaving school and a lot ... are middle-aged or over . . . [but] even 
if they can 't read and write it doesn 't exclude them from doing 
it because ... it 's on what they do every day. 
internal verifier, Social Care 
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Section 4.2 


Definitions of progress 



Progression is conceptualised as both a ‘horizontal’ aggregatory 
process and a ‘vertical’ developmental process. In turn, both 
of these categorisations can also be subdivided into what 
might be termed intrinsic ‘progress’ - with respect to learning 
(or towards a learning goal) - and more extrinsic ‘progression’, 
with respect to moving from one accomplished achievement 
or qualification to another. 

Within tracks, awards and levels, there is the aggregatory 
acquisition of modules/evidence at the same ‘level’ of difficulty 
(eg NVQ Level 2, AVCE units) However, because there is no clear 
change in marking criteria, some tutors do not recognise this as 
manifesting either ‘progress’ or ‘progression’: 

Students are full-on from day one. Every unit contributes to UCAS 
points, so they have to be full-on from the beginning. But the 
programme is not stepped, so the level is supposedly the same 
throughout. This is a real issue, a real problem. How do you put 
progression into that? 

AVCE Sp&L tutor 

There is also accumulation of credits and awards across subjects 
within the same level of difficulty (eg Literacy and Numeracy in 
basic skills testing) Success in these programmes then maps 
onto progression in others, though whether this ‘progression’ 
signals actual acquisition of knowledge and competence, as 
opposed to simply gaining accreditation for access to another 
course, is a moot point: 

They need to get up to Level 2 if they are on a course that requires 
their GCSE A-C and they’ve not got it. 
basic skills tutor 

We’re all there for one reason. We’re all there at the end of the day 
to pass it ... I just want to get it over and done with. Get it passed 
and carry on. 
pre-access student 

Aggregatory accumulation can carry problems with it if the 
amount of evidence required within a level is too great. Many 
examples of non-completion (= failure) were encountered as large 
numbers of units were completed by candidates, but without the 
full number for an award ever being achieved: 

If you're going for Level 3 which the bulk of ours do, you don’t need 
to do Level 2, so you could end up with a youngster starting doing 
three years’ training and then part of the next year finishing off the 
assessment so it’s three and a half years and they haven't got 
anything ... if ... they can go home and say ‘Look mum, look dad, 
look boss...’ and say, ‘Well I've got some units, I’m making some 
progress. . . ' they've got their self-esteem. 

MVE lead verifier 




You’ve got a unit like diagnose complex systems, repair complex 
systems, which means that to diagnose something, you could 
be looking at brakes, lights, steering, engine, transmission, 
so you're never going to finish a unit... That’s bad news because 
if somebody says well I've done brakes and now I've done 
a module on lighting and electronic systems you can actually see 
a progression and I think personally success breeds success and 
if they see they've only got a few left to go ... whereas if they've 
been working two and a half years and they’ve achieved nothing 
it’s really hard. So I think the way that the Standards are written 
has got a lot to answer for. 

MVE external verifier 

Such problems can also be compounded by the attitude of 
employers and training agencies wanting to save money on fees 
so that candidates are only entered for the most advanced 
qualification available to them: 

I’m not allowed to get my Level 2 because [the training agency] 
won’t pay for it. They’ve put me on a Level 3 course and even 
though I’ve done the Level 2, I’ve done the portfolio, I’ve done the 
exams and passed them all, there’s only one thing stopping me 
having my Level 2 and that’s getting the certificate. And [the 
training agency] won ’t pay for it and they said we put you on a Level 
3 course so you only get it at Level 3. You can’t get your Level 2... 
MVE apprentice 

Lack of accumulative progression is a key issue underpinning 
the development of NOCN accreditation in some adult education 
settings, where the acknowledgement of even small positive 
steps is considered to be extremely important: 

The point is that OCN is a sort of can-do system, you know, you’re 
always sort of adding something on, you’re gaining something. 
Access tutor 

This is clearly an issue that NVQs need to address more generally, 
and changes to introduce more unit accreditation are under 
discussion by QCA and awarding bodies (cf. the QCA ‘Framework 
for Achievement’ noted above) Equally, however, even within 
an accumulative system, lack of grading can produce similar 
sorts of problems, as some candidates feel they are operating 
within a very wide achievement zone and not being challenged 
(or encouraged) to progress within it: 

I always achieved Level 3 but I didn 't have any input to build on 
that. . . i was pretty bored towards the end. 

Access student, now an undergraduate 
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Within tracks/awards there is also a sense of ‘upward’ movement 
through levels of difficulty (from Level 2 to Level 3, or AS to A2) 
However, the evidence is mixed on whether or not there is any 
actual change in the level of difficulty encountered. Awards 
claim that there is - eg from ‘description’ to ‘evaluation’, or 
accomplishment of an activity to supervision of it, etc - but it is 
difficult to identify clear empirical evidence that real individuals 
find different tasks at different levels more difficult to complete - 
in other words, that there is a progressive increase in the nature 
of the cognitive challenge. Some respondents indicated that the 
level of workplace challenge was often greater than that required 
by the NVQ, or by supposedly superior courses: 

One of the things that amazes me, on the key skills, for a motor 
mechanic it’s IT Level 1 and then it’s number and communications 
Level 2. 1 would have thought the IT would have been at Level 2 or 
even 3. They’ve got to know how to use this IT equipment. The 
car’s got a memory as well. So we need people with reasonable 
intelligence to work on these cars. 

FE college head of MVE department 

[AVCE is] more practical, more geared to coaching. Some people 
say it is easier [than A-level], but I think it takes more guts to stand 
there coaching with kids. 

AVCE student 

With respect to returners to learning, attempting to come to terms 
with and progress through basic skills tests, the very earliest 
encounters with even ‘Entry level’ work could actually be more 
challenging than later encounters because of the social 
challenges involved: 

They are worried about things like getting the kids to nursery, 

[and] ‘How am I going to pay the gas bill, the phone bill, ’ whatever. 
‘I’ve got to go to work, do a 10-hour shift and then come here, ' 
so it’s all those other things. 

Adult basic skills tutor 

It’s fitting it in. I mean I’ve only got one [child] ... [but] ... for me 
it’s very hard ... trying to explain to them that if I’m sitting on 
the computer working, leave me on the computer. I don’t want to 
get up and do other things. That to me is getting to killing point 
at the minute which is the hardest bit. Getting peace. I don’t know 
how you do it... 

Access student 

In these instances, it is very difficult to distinguish between 
cognitive and affective challenges. Similarly, difficulties 
encountered in moving through levels in vocational training can 
relate to lack of access to resources and lack of opportunities 
to display competences, rather than the challenge of the task 
per se - see Sections 4.5 and 4.8, below. 



With younger workers moving through more distinct levels of 
academic as well as work-based progression, the challenge may 
be more apparent: 

Level 2 is very much worksheet-based with some exams. Level 3 
there is a big jump for them . . . they go from just filling in 
worksheets and really quite simple multiple-choice type of exams, 
short-answer exams up to them writing assignments and exams in 
every single unit as well. . . They are given assignments for the first 
time. They are asked to go away and research information on their 
own for the first time. 

Sp&Rec Progression Award college tutor 

Progress in written work is paralleled by increased difficulty in 
practical contexts: 

For Level 2 they plan, deliver and evaluate one session. . . 

For Level 3 they have to plan a six-week coaching session . . . and 
it can’t be delivering to their peer group, it must be an external 
group, so that’s a big sudden jump. 
college curriculum manager 

Interestingly, given our previous discussion of the problem of 
gaining a ‘distinction’ in AVCE, the Sport and Recreation 
Progression Award at Level 3 (ie AVCE equivalent) not only involves 
a major step-up in terms of academic and practical challenge, but 
also seems to involve reduced levels of tutor/assessor support. 

It was reported that the external verifier associated with the 
colleges visited for Sport and Recreation did not allow criteria and 
feedback to be shared with Progression Award candidates at 
Level 3 - see Section 4.4, below. Once again, what was seen as 
‘help’ was not allowed. Somewhat ironically, however, as we shall 
also see in Section 4.4, A-level tutors do give students help in 
terms of exam practice and coaching, and indeed - even within 
A-level - some tutors spoke about some students being ‘workers, 
not academics’, who achieve good grades through ‘sheer graft’. 
Thus the AS/A2 structure would appear to assist such students 
in building incrementally towards success even as AVCE and 
Progression Awards eschew too much tutor input for the highest 
achievement to be awarded. 

Different types of activity can certainly be perceived as more or 
less difficult, however, eg across the ‘academic’ or the ‘practical’ 
tracks from AVCE to academic A-level or Access courses to HE. 
Most respondents had of course ‘crossed the tracks’ in the 
opposite direction, from academic school failure to vocational 
education and training. In this respect they thought their 
endeavours became ‘easier’ as they became more practical 
and more suited to what they thought were appropriate to their 
interests and abilities. For some this movement finally took 
place after 16+ as they discovered that AS was ‘not for them’ and 
they moved onto to AVCEs (or BTEC) rather than A2s. This was 
seen as ‘sideways progression’, but still progression, insofar as 
a positive decision had been taken. The elision of progression 
with decision-making is worth noting here: individual learners felt 
they were making progress by taking (what they considered to be) 
positive decisions. 
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‘Upward’ progression across tracks was seen in terms of moving 
from the ‘practical’ to the ‘academic’, after having benefited from 
a ‘second chance'. This occurs to some extent in the move from 
AVCE to HE and via specific ‘Access to HE’ programmes, but these 
‘progressions’ still tend to be within broad vocational tracks - 
from AVCE Business Studies to degree-level Business Studies, 
from ‘Access to Nursing’ to a Nursing degree. This could be seen 
as another example of progression within tracks but through 
higher levels. However, such easily identifiable ‘stepping stones’ 
do carry positive messages for aspiring learners who may have 
failed previously: 

In the past I’ve tried GCSE and I’ve come out with the same result 
every time, and with me working as well full-time, I think it’s best 
just to pass this [pre-Access course] and get myself the pre- 
nursing and then the nursing and university, so I know what I’m 
aiming for. 
student, pre-Access 

Progression was also reported from specific ‘Access’ courses 
(eg to Social Work) to more general undergraduate studies 
(eg Humanities), and from NVQ to professional (nurse) training 
which can be construed as a more explicit move to a different sort 
of learning and assessment environment: 

NVQs have been well received, and people have often gone on 
from doing NVQ to ... progress further, and some of them are now 
doing nurse training ... where there’s close links made between 
certain sectors and their local universities, that’s worked well... 
awarding body product manager 

The trainee assistant practitioner role Level 3 that we’re 
developing is also linked to a Foundation degree. It’s a two-year 
programme so they’re studying for the diploma and simultaneously 
they have to get a Level 3 in Care. At the end of that second year 
they can go on to do another degree, another route/pathway. . . 
internal verifier, Social Care NVQ 

Take Helen, for example, she started off as an admin assistant ... 
then gradually took things on and was doing a Level 3 ... and now 
she 's a graduate with a really good honours degree and is a senior 
probation officer. . . 
lead verifier, Social Care NVQ 

This third quote, however, and to some degree all three of the 
above quotes, while demonstrating progression, also endorse the 
‘academic track’ as the one to aspire to, the top of the hierarchy. 

Progression is also identified in the data as moving from 
education/training into employment - progression to full-time, 
qualified employee - and there is a clear change of status 
implied here, as well as a change of level in terms of operating 
competence. Achieving well at work, especially ‘holding down 
a job’, is important, and brings a sense of achievement and 
progression for young apprentices in a new environment. Similarly, 
progression was encountered within employment as awards 
were achieved. 



Linked to all of the above examples is a notion of progression as 
social mobility. Other examples include the move from NVQ-taker, 
to NVQ trainer/assessor - ie a movement from an operative to 
a supervisory and training/assessing role - and from Sure Start 
participant to Sure Start facilitator and organiser. In the 
social care sector in particular, a career ladder that didn’t exist 
previously has been created. Many care staff who have 
demonstrated some competence and ambition have changed 
their jobs after completing their Level 2 certificates and 
now work as assessors in training agencies. 

Further examples in the data include movement into HE from 
Access courses, with a clear implication of social class 
progression - ‘bettering oneself, ‘doing something’ with one’s life. 
This occurred especially with respect to female returners on 
vocationally-oriented programmes such as Access to Nursing and 
Teaching but also with respect to more academic programmes 
such as Social Science degrees and which, as we have seen 
above in at least two AVCE tutors’ cases, led into college teaching. 
Such personal experiences feed into the ‘second chance’ 
perspective very directly and reproduce it in situ and in action: 

I went through a divorce four years ago... I was ... raising three 
children on income support . . . and I just thought right if I don 't go 
out and do something then I’m never going to be anything. 

Access student 

A lot of it is because it’s been triggered by my job, you see, I work 
in the NHS and ... I’ve watched a lot of people go on and do better 
for themselves and I’ve kind of still been the same and it makes 
you think doesn ’t it. And that’s why I did it really. 

Access student 

Progression can also be conceptualised in terms of acquisition 
of social capital - personal confidence, social engagement, 
new or increased personal networks, community development 
and vitality - with more ‘choices’ available to individuals. Such 
progression is very apparent in the adult and community 
education sector in particular, though very difficult to ‘capture’ 
in terms of specific achievements and outcomes (for individuals 
and institutions alike). 

My daughters say I’m doing very well, because I couldn’t even 
write out a birthday card or Christmas card or anything. I’m doing 
all that myself now. Full of self-confidence in myself now. Feel a lot 
better about myself. Before, I wouldn't talk to anybody... And I’ve 
got 14 grandchildren. ..I’m doing a bit of [reading to them ] now, 
because I mind [some of] them three days a week. 
basic skills learner 
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Sometimes the very act of participation is enough for learners, 
and they may even resist assessment success (the policy-makers’ 
measure of success and progression) to remain within the 
learner community: 

A lot of comments I’ve heard, especially from the ladies, is that 
‘I’m stuck at home all week and I really enjoy coming to the 
college, my English is improving and I’m meeting lots of people’. 
And for another group, asylum seekers, because they’re brought 
here to this area, they know absolutely no-one in the area, so they 
do make friends through the college and they do become happy. 
basic skills tutor 

If I ever got a job, maths and English would help me. But I don’t 
want a job! It’s a hobby. Better than watching TV. 
basic skills learner 

The ‘wider benefits of learning’ are very much part of the ethos 
(and the observed reality) of the sector, but for the most part are 
not explicitly pursued or recorded within formal assessment 
discourses and procedures although the LSC is reviewing this in 
its development of New Measures of Success (LSC 2005). 
Interestingly enough, of course, such benefits also accrue, and 
probably even more substantially, to what one might term 
traditional, high-achieving A-level takers, but such benefits are 
rarely discussed - they are taken for granted as a corollary of 
academic progression. 




As noted in the introduction, this research was commissioned in 
the context of a growing concern with respect to ‘over- 
assessment’ in the English education system, along with a 
growing policy interest in formative assessment. A key issue with 
respect to LSS policy-makers and practitioners is the possibility 
that external tests and exams will be particularly disliked or 
responded to negatively by post-16 learners in further education 
and training, including adult learners, given their probable lack of 
prior achievement at school. Such a view was certainly expressed 
by officials in the sector: 

As you can imagine, with the kind of client base or candidate base 
for this, lots of people who haven’t done that well at school, 
haven’t been at school for years, and who are women who have got 
busy kinds of jobs and families, do not like external tests, and so 
there is resistance within the sector to actually having external 
tests. 

chief certifier, Social Care NVQ 

However, we do not actually know whether or not this is true, and 
the data do not confirm it, at least not in any straightforward way. 
Some learners, especially older learners, certainly do recall very 
bad memories of testing: 

Like the exams, you just put your name on the top and walked out. 
You never sat any exams or anything. .. To be honest, years ago . . . 
the mere mention of an exam frightened me to death, bring me out 
in cold sweats... 
basic skills learner 

But a key finding from the study is that assessment methods 
per se do not inhibit achievement and progression in the sector, 
at least not of those learners continuing within the system, 
and hence available as respondents in the study. Assessment 
methods in and of themselves are not considered particularly 
important to achievement and progression by learners, once 
choice of award (and hence academic/vocational track) has been 
made. All of the programmes under study except forA-level 
Business Studies (100% external examination) and Social Care 
NVQ (100% portfolio) include some combination of coursework, 
practical assessment and tests and most learners, certainly the 
younger ones recently out of school, thought this was appropriate: 
‘I can’t see how you would assess it differently really.’ Indeed, 
even among adult returners there was a resignation, and in some 
cases even positive appreciation, of the fact that exams had to be 
faced and could bring benefits in terms of self-confidence - as 
long as you passed: 

It's not the most wonderful thing to do ... I felt very very nervous 
but when it was over and we got our results I 'm glad I did it 
because it is nice, it is much nicer to actually do an exam and know 
that you did get that information in your brain and that you were 
able to transfer it onto paper rather than just being sort of tested 
or given your mark on the things that you've handed in because 
you had masses of time to do that. You get more achievement from 
having done the exam. 

Access student 
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You work through each unit and you take your time ... make sure it 
is correct, and you get to the end and do the final test . . . when you 
do the test it gives you confidence. . . 
basic skills learner 

Assignments you can take your time. . . So exams . . . give a true 
idea even though I don’t enjoy them. 

Access student 

I found that easy because it’s over and done-with within two hours. 
All they're asking you to do is to demonstrate what you know in two 
hours and then once you walk away from that exam you can forget 
about it then, can’t you. So yes, I enjoy the exams. 

Access student 

Thus despite strong images among awarding body officials and 
some staff in the colleges, that adult and vocational students 
‘don’t like exams’, learners were much more instrumental and 
strategic in their views. There was no uniform or strong preference 
for coursework or exams. Although some students actively 
disliked exams because ‘all the pressure comes in one go’, 
there was also a view that they ‘got things out of the way’ and 
demanded less time than coursework. AVCE students doing AS 
qualifications alongside their AVCE offered some interesting 
insights about why exams and coursework were an acceptable 
combination. Some AVCE students liked ‘having a mix, doing 
things on the day and also doing things in real-life’ and liked 
exams because ‘you get them over with, they don’t keep dragging 
on like assignments do’. The overall framework of the AVCE regime 
allowed students to predict their grades and aim for what they 
wanted, high or low. And in both cases, AVCE and AS, exams 
were not regarded as particularly stressful because they could 
be retaken. 

In effect, learners self-select awards (within the boundaries 
set by prior achievement) likely to suit their approach to learning 
and assessment, and privilege the combination of methods 
with which they feel most comfortable. Additionally, there is 
clear evidence from assessors/tutors and candidates alike that, 
while a broad range of assessment methods is thought to be 
appropriate in terms of fitness for purpose and validity, externally 
set and marked tests and exams are regarded as higher status, 
irrespective of validity issues, and hence a more valued indicator 
of achievement: ‘You need exams, everyone has to do them 
at some point,’ (AVCE student). Thus while exams might generally 
be thought of by the majority of candidates in the study as difficult 
and ‘not for them’, specific, small amounts of external testing 
is accepted as appropriate and inevitable and hence, to reiterate, 
not an issue perse: 

Bit of a pain but got to be done. 

MVE trainee 

Well I chose to do it, so I have to do it. 

MVE trainee 

There’s the unit exams at end of each unit - it’s too much - I’m not 
very happy about that but you've got to do it haven’t you... 

Sp & Rec trainee 



Too much external testing is both disliked and avoided, but not so 
much because of the method as such, rather because of its 
association with the academic track and more general problems 
of the perceived appropriateness of the award/programme: 

I’d rather do workshop tasks or something but it’s got to be done. 
MVE trainee 

I was scared because I didn 't really like school anyway. I hated 
it ... I just didn 't like it at all and coming here I thought it's going 
to be like school again . . . and the first lesson . . . really took me 
back ... [but] the whole introduction towards college made me feel 
this isn’t so bad, this is alright you know. 

Access student 

This issue of past experience and perceptions of testing is also 
illustrated by one incident of workplace basic skills teaching and 
testing. An innovative programme was negotiated with 
management and unions in one particular workplace, and 
included some initial diagnostic assessment: 

[Named] Foods is a great example. We started a big project there 
over a year ago, and there were four of us walked in there with 
boxes of paper- and straight away the unions jumped on it, and 
the manager said, ‘They are not doing that, they are not being 
tested like that!’ And I had to go in to them and chat to them and 
explain. 

basic skills tutor 

All this suggests that it is not sensible to see assessment 
methods in isolation, they are not reducible to matters of purely 
technical or pedagogical decision-making. They come with the 
‘territory’ of a regime, a type of qualification, a programme and a 
personal identity, and are therefore bound up in defining 
qualifications as positional goods which different individuals 
pursue in different ways. In this respect, learners develop 
‘assessment careers’ in much the same way as (and in tandem 
with) ‘learning careers’ (Ecclestone & Pryor 2003). A-levels 
without any - and indeed without a large weighting of - external 
exams are almost unthinkable. Equally, too many externally- 
designed assessments would not be appropriate or acceptable 
within more vocational tracks, though they are sometimes 
insinuated into such tracks by policy-makers in violation of fitness- 
for-purpose issues - see Sections 4.7and 4.8 below on key skills 
and validity). 
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Assessment method(s) ‘smorgasbord’ 



In the questionnaire, questions were asked about learner 
preferences for different assessment methods. While the contrast 
is marked between, for example, A-level takers’ responses to 
questions about assessment methods (even in vocationally- 
oriented subjects such as Business Studies and PE), and those of 
NVQ takers, the implications of the contrast are more subtle. 

When asked to rank preferences, A-level takers identified project 
work, written assignments and external exams as their top three 
(in that order) by numbers of respondents selecting them: all very 
much ‘written’ formats. On the other hand, NVQ takers identified 
online tests, observation by assessor and practical tests as their 
preferred three: all very much non-written forms - see Appendix 3. 
Thus A-level takers ranked external exams only third in their 
expressed preferences, while NVQ takers ranked other forms of 
testing first and third. From this it would seem that it is not so 
much testing as such that worries learners, but rather its context 
of operation and its association with extended writing and the 
transformation of practical knowledge into discursively presented 
propositional knowledge. 



7 NB the high ranking of online tests is 
probably skewed by the larger number 
of MVE apprentices returning 
questionnaires compared with other 
groups. Nevertheless, it is clear that 
they are, for the most part, liked by 
those who have encountered them, 
including in adult Basic Skills testing. 
This is discussed further in Section 
4.7, below. 



Perhaps surprisingly, multiple-choice online tests seem especially 
acceptable - see also Sections 4.6 and 4.7 on e-assessment and 
basic skills testing. 7 Certainly in ‘return to learn’ situations, while 
testing can cause concern, so can the extended writing 
associated with assignments: 

It is hard. Like this two weeks we’ve got four assignments in and 
they’re all quite big ones at that. 

Access student 

And if it’s been a long time since you were at school, you’re well 
out of practice and everything’s completely new and it’s scary 
isn 't it. . . 

Access student 



It was having them all at the same time . . . they all had to be in and 
that’s very hard, trying to do one and then you almost had to run 
them along with each other to get them all in. 

Access student 



The questionnaire results 

A vast range of assessment methods are used in awards in 
the LSS. Learners identified the following, ranked according to 
percentage of learners, responding that they had experience 
of the method in their courses and training. 



Table 1 


N 


Percentage of all returns 


How are you assessed? Wrjtten assignment 
(Tick all that apply) 


224 


86% 



Observation by assessor 


138 


53% 


Presentations 


137 


53% 


Key skills test 


133 


51% 


External exam 


112 


43% 


Group work 


112 


43% 


Project work 


106 


41% 


Internal exam 


100 


39% 


Practical test 


95 


37% 


Spoken questions/interview 


87 


34% 


Online tests 


80 


31% 


Simulation/mock situation 


69 


27% 


Witness testimony 


62 


24% 


Video/audio recording 


14 


5% 



Learner preferences were expressed as follows: 



Table 2 


Assessment method 


N 


Mean score 


How would you prefer 
to be assessed? 

(List top three in rank 
order, ie 1 = highest 


1 Written assignment 


133 


1.53 


2 Online tests 


44 


1.77 



ranking) 

Analysed by mean 
score 








3 Practical test 


80 


1.86 



4 Observation by assessor 


59 


1.95 


5 External exam 


52 


1.98 



6 Project work 



73 



2.21 



Table 3 

How would you prefer to 
be assessed? 

Analysed by numbers of 
students who chose the 
type of assessment as 
one of their top three 



Table 4 

Rank order of forms 
of assessment 
preferred by candidates 
(Choose top three 
methods in relation 
to experience) 

(260 completed 
questionnaires 
returned from 
890 distributed = 

34% return) 
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Assessment method N 

1 Written assignment 133 

2 Practical test 80 

3 Project work 73 

4 Observation by assessor 59 

5 External exam 52 

6 Group work 47 



In each case (in Tables 2 and 3) ‘external exams’ come well down 
the list in fifth place, buttesting as such is not responded to 
negatively, with online tests second in Table 2 and practical tests 
second in Table 3. 

It is also the case that these figures need to be linked to 
experience of assessment methods and possible over- 
representation of sub-samples in the total. For example, 44 chose 
online tests as one of their top three methods of assessment, but 
only 80 report having experience of it, virtually all of whom will be 
MVE apprentices who prefer online testing over paper and pencil 
testing; hence online testing is highly ranked in Table 2, but not 
Table 3. 

Thus when comparing assessment preferences, in tandem with 
experience of methods, the following rank order of the most 
popular forms of assessment is produced: 



Assessment method 


N = 

experience of 


N = choosing 
1 of preferred top 3 


Percentage 


Practical test 


95 


80 


84% 


Project work 


106 


73 


69% 


Written assignment 


224 


133 


59% 


Online tests 


80 


44 


55% 


External exam 


112 


52 


47% 


Observation by assessor 


138 


59 


43% 


Group work 


112 


47 


42% 


Spoken questions/interview 


87 


33 


38% 


Presentations 


137 


44 


32% 


Internal exam 


100 


28 


28% 


Simulation/mock situation 


69 


18 


26% 


Key skills test 


133 


11 


8% 


Video/audio recording 


14 


1 


7% 



Witness testimony 



62 



2 



3% 



Practical tests are most preferred, while witness testimony is far 
and away the least preferred method. There are also some major 
differences between sub-sectors, such that although 
‘Presentations’ are not generally disliked, they are very unpopular 
with some adult returners, which in turn can lead to resentment 
from others: 

20 minutes. Bloody hell. I don’t want to do that, it’s scary. 

Access student 

I’m petrified. I don’t do speaking out loud. 

Access student 

A lot of us were in a right state because of doing the presentations 
and some people we know didn't do them because of excuses and 
I think really that’s wrong because it’s one rule for one and one for 
another. 

Ex-Access student, now undergraduate 

Also interesting is that while key skills tests are unpopular (8%), 
online tests (sometimes thought of as co-terminous with KSTs) 
are considerably more popular (55%). In fact, online tests also 
include ‘underpinning knowledge tests’ and adult Basic Skills 
tests, and, while these are generally undertaken willingly, key 
skills tests are particularly resented because of the imposed 
necessity to undertake them. So it is this imposition, rather than 
the tests or their format as such, that leads to their unpopularity. 
This is discussed further in Section 4.7, below. 

In light of these findings, a key policy issue must be how decisions 
about various combinations and weightings of assessment 
methods are made by awarding bodies, and in particular product 
managers and award developers. The most legitimate response 
is ‘fitness for purpose’ - varieties and combinations of methods 
are needed for the valid assessment of different abilities 
and competences, and particularly of competence in situ. 

But attention to learner preference might also be considered, 
especially when thinking about maximising achievement 
and progression. 

A complicating factor here is what we might term ‘methods 
in action’ rather than ‘methods in principle’. Thus for example 
observation by assessor is a more preferred method (43%) 
when compared to simulations (26%), but less preferred when 
compared to practical tests (84%). Given that much of our data 
indicate that observation by assessor often becomes, de facto, 
a simulation in action, because candidates do not have access 
to the full range of workplace activities and assessment 
opportunities, and scenarios have to be enacted for the 
assessor’s visit - see 4.8, below - perhaps a series of explicit 
‘practical tests’ would be a better solution. Variations in task 
would be controlled by the test-setter, and such evidence as we 
have suggests that learners would prefer practical tests, certainly 
when compared to simulations. Interestingly enough, a version of 
this voluntarism already exists in A-level and AVCE courses, 
where assignments can be submitted in lieu of certain module 
tests (or vice-versa); likewise in Access programmes where 
assignments can replace exam credits: 
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There were big assignments with quite a few credits and then 
I didn 't have to take the exam ...I thought I don 't need to take 
the exam because I’ve got enough credits, why should I put myself 
through this, so I didn 't take it. I had the credits and I passed 
my Access. 

Ex-Access student, now undergraduate 

Say for example you had psychology where you have a written 
assignment for the first one, a written assignment for the second 
one and then a presentation for the third one and an exam for the 
fourth one. If they say, I still can’t do presentations they can, of 
course, drop that and just get 3 credits in psychology without really 
damaging their chances of their certificate too much... 

Access tutor 

On the face of it, allowing vocational learners to substitute a 
practical test, for a simulation, or vice versa, and indeed other 
assessments as appropriate, seems no different in principle from 
the above examples. (NB: this lack of consistency in the way 
assessment regulations operate across ostensibly comparable 
sub-sectors of the LSS, can also be observed with respect to key 
skills, discussed in Section 4.7, below.) 



Section 4.4 



Supporting candidates: coaching, practising and eliciting 
evidence - ‘assessment as learning’? 



There is a significant, even overwhelming, culture of support for 
learners/candidates at every level and across every sub-sector of 
the LSS. Tutors, supervisors and assessors alike take their 
responsibilities to promote learning, engender motivation and 
encourage achievement very seriously. This is very much where 
the ‘second chance’ perspective is manifest: 

There 's somebody worse than me in there and some are better 
than me. But you don’t feel ashamed in there because we’re 
all in the same boat. Nobody’s slagging you off like they did 
at school. . . We help one another best way we can. . . Because you 
know everybody and they’re in the same boat as you. . . It is like 
a social club. 
basic skills learner 

Even at A-level, teaching support is provided through the breaking 
down and interpreting of assessment criteria, and the involvement 
of tutors in formal examining and moderating roles for ABs, which 
helps to develop their understanding of the assessment process 
and which, in turn, they can pass on to students through exam 
coaching. 

AVCE and A-level 

This culture of support is apparent in choices of awards and 
awarding bodies even before tutors begin to provide detailed 
guidance on assessment tasks and criteria: 

We have changed exam board - from [AB1 to AB2]. The practical 
with the former was too long-winded and complicated, and the 
moderation system was complicated ... [ the AB2 syllabus ] is also 
a little bit more compartmentalised ... which seems to suit our 
pupils. With [AB1 ] the questions went down the synoptic line, 
which our kids found confusing. . . 

AVCE Sp&L tutor 

ABs also came in for criticism about lack of responsiveness to 
queries, and clearly this will influence choice of AB: 

[They] are very good at telling us when we are doing something 
wrong. . . Don 't try ringing them for an answer - 1 Ve given up doing 
this. There is a lot of unevenness, depending on who you get... 
AVCE Sp&L tutor 

[AB3] move the goalposts regularly. If you have any queries they 
are a bit airy-fairy. They sometimes don ’t get back to you. 

AVCE Sp&L tutor 

[AB3] are chaotic. . . They lose things, they don 't always keep up 
with simple logistical matters . . . [and] don 't ring them up with 
a question -you’ll get an Australian or a Kiwi on work experience 
who won ’t know how to help you. 

AVCE Sp&L tutor 
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8 Savory, Hodgson & Spours (2003) 
similarly report a move away from 
AVCE towards offering BTEC 
Nationals, and note widespread 
dissatisfaction with ABs responses to 
queries. They quote one respondent 
complaining that ‘the exam boards 
have become so big that they are 
drowning and we find it almost 
impossible to find a real person to 
talk to about new specification 
problems.' (p!6) 



In some colleges in the study, curriculum managers have been 
returning to BTEC programmes, which they feel are more genuinely 
vocationally-oriented, rather than AVCEs. Interestingly enough, 
given our earlier discussion of the different cultures of 
assessment tracks, although BTEC is part of the merged Edexcel 
awarding body, it is retaining its vocational ‘brand’ as a distinct 
marketing device. Additionally, BTEC students are not counted in 
national achievement data for Level 3. Thus in one of the colleges 
where this change was taking place, the overall pass rates for 
AVCE, while still good, were not as good as A-level, and ‘pulled 
down the college’s results’. Taking vocational students out of the 
data obscures the college’s attainment of fewer higher grades 
in AVCE compared to general A-level. This local decision reflects 
a national discrepancy in achievement between the two 
qualifications, but it also illustrates how measures of 
achievement can be constructed in different ways by diverse 
bodies and interests in the education system. 8 

Once ABs and syllabuses are selected, detailed grade criteria are 
articulated for learners at A-level and AVCE - the breaking down of 
awarding body guidelines and instructions into detailed mark 
schemes, assignment plans, etc. for students to follow: 

One of our problems is with our pupils’ language and literacy. They 
can tell you, in their own way, but at A-level [the examining boards] 
want to hear technical terms and expanded vocabulary. . . 
[Students] need to be a little more analytical at A-level... So we 
have drafted a crib-sheet with words, explanations. 

A-level PE tutor 



We have spent a lot of time . . . coming up with a sort of a template 
to issue to our students as a starting point to give them something 
to work on ... writing frames, templates to fill in, bullet points to 
follow... 

AVCE Sp&L tutor 



In turn, students can draft and re-draft assignments, receiving 
feedback on strengths and weaknesses and what needs to be 
done to improve the grade. They can also re-take unit and modular 
tests as necessary to improve grades. Sometimes tutors operate 
with the ‘straight’ AVCE/BTEC nomenclature of pass, merit and 
distinction; sometimes they operate with a range of grades which 
parallel AS and A2 (grades E-A) and which in turn ‘map onto’ AVCE, 
thus E/F refer to ‘describing and identifying’ and would 
correspond with a pass; D/C involve ‘understanding’ and ‘bringing 
together’, and correspond with a merit; A/B focus on ‘critical 
evaluation and analysis’ and correspond with a distinction. When 
asked what an A grade meant in AVCE Business Studies, one 
student responded: 

It’s analyse, evaluate, and stuff like that. You have to explain 
things but more in-depth. Instead of just summarising, you have to 
extend it and make it relevant and link it to what you’ve already 
put, it’s clear and it flows and it’s fully described. Not just going off 
on a complete tangent. You have to include what they ask, not let it 
get too theory-based. Mind you, actually doing it’s another thing. 
(Laughs) 



Other students were equally well-tuned to the criteria and drafting 
process: 

At the start of each module we get like a sheet of paper, it’s got 
‘for an E you have to do this, for a D you have to do that’. . . They tell 
you what you have to do to get a good mark. 

AVCE student 

You do a draft, you get it back and you have a week to change 
things and hand it back again . . . can improve it - that’s good when 
we don 't know what we are doing! You read their comments and try 
to do what they say. 

AVCE student 

Tutors worked very hard to make their feedback as detailed 
as possible: 

/ talk through the assessment criteria grid with them and the 
assignment brief, pinpointing the relationships between P, M and 
D [pass, merit and distinction] and that it does evolve through to 
D. [Some] students like to go for the best grade possible and 
discuss how they could go about getting an M. There again, some 
students just aim for a basic pass. . . Then I see a draft work, read 
through it, make notes, talk to each one, show the good areas 
in relation to the criteria and explain why and how if they have met 
them, saying things like ‘You’ve missed out M2...’ Some will 
action it, some won ’t. 

AVCE BS tutor 

The process of ongoing assessment means that we use a lot of 
verbal feedback and we give them formal interim feedback too ... 
they have time to improve and develop and then they must submit 
formally at a cut-off point. Students find that motivating, they want 
feedback, it helps to reassure them. 

AVCE BS tutor 

I assess continuously, especially with sport skills... Also there are 
summative assessments [written tasks that respond to a specific 
question and a set of criteria]. These are always worked on in a 
way where they can improve their grades, and they get a lot more 
than one chance . . . you can see a piece of work three times. If 
there is a large group, this gets out of hand. But I want to get the 
best I can for my students. 

AVCE Sp&L tutor 

At A-level, in both colleges where Business Studies was the focus, 
tutors had chosen a syllabus with 100% external examination. In 
one college, however, students could opt to do one unit as either a 
coursework project or an examination and one student chose both 
options so that she could pick her higher mark to go forward for 
final grading. Students also take three exams in the first year AS 
qualification, but can retake poorly-graded papers or simply not 
progress to Year Two if grades are poor: 

They can retrieve poor grades [through re-sitting module tests]. 
A-level BS tutor 
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Thus ‘good teachers’ are those who can handle this workload and 
schedule this formative feedback within realistic time-scales, 
offering clear guidance that students feel they can follow. A-level 
and AVCE alike involve a great deal of criteria-focused ‘coaching’ 
of students. The potential downside of such activity is that 
achievement can come to be seen as little more than criteria 
compliance in pursuit of grades. Moreover, while the pressure of 
coursework assignments can become intense, the responsibility 
for putting in the ‘hard work’ of assessment in pursuit of 
achievement might now be said to fall as much on the shoulders 
of tutors as on the learners, and a great deal of ‘hidden work’ is 
undertaken according to tutor disposition (cf. also James and 
Diment 2003): 

The volume of marking we have ... 60-70 assignments every few 
weeks, formal ... and informal ... [feedback]. It’s one of the best 
ways for them to learn, but there’s no time or recognition to that 
marking - it’s hard work. 

AVCE BS tutor 

Always in every group, you have students who are very weak, 
perhaps only just got onto the course or ... who can 't organise 
themselves. For them, coursework is a nightmare. I've got two in 
particular who are not passing any of their three units; we 're going 
to start assignment surgeries where they will get one-to-one 
support. 

AVCE BS tutor 

In a very real sense we seem to have moved from ‘assessment of 
learning’ through ‘assessment for learning’ to ‘assessment as 
learning’, for both learners and tutors alike, with assessment 
procedures completely dominating both pedagogy and the learner 
experience. 

The exception to this level of assessment transparency and 
culture of tutor support is in the college-based Sport and 
Recreation Progression Awards. Here it was reported by the 
course tutors that the external verifier for the two colleges 
involved in the fieldwork insisted that AB criteria could not be 
shared with candidates, and as we have seen earlier in the report, 
the move from Level 2 to Level 3 thus involved more difficult tasks 
being undertaken by learners but with less direct tutor support.: 

When it comes to the progression award ... the marking criteria for 
tasks for the progression. . . In NVQ they're allowed to see that, 
with the progression award they aren’t ... the students are left to 
their own devices, because of the instruction from the El/... If a 
tutor gives them a lot of support, the maximum they're allowed to 
get is a pass ...as regards the assignment. .. It's very difficult to 
say that a student has done an assignment with no support at all 
from a tutor... That's why for assignments we limit to an absolute 
minimum the support the student gets. 

Sp&Rec curriculum manager 

In some respects, this might be said to parallel the ‘objective’ 
approach of A-level, but in others it might more correctly be said 
to be a parody of A-level, which, as we have seen, actually involves 
a good deal of detailed help with understanding assessment 
criteria, re-taking unit tests, and so forth. 



Support in the workplace 

This injunction applying to the Sport and Recreation Progression 
Award does not appear in the AB handbook and seems to be an 
entirely local interpretation by the external verifier, but it certainly 
illustrates the ambivalent status of technical, vocational awards 
delivered in college rather than workplace settings. ACVE and 
A-level students get support from their tutors. So, it transpires, 
do learners in the workplace: 

Basically the assessor stands to the side... All she wants to do is 
to make sure that you’re doing it right [and] if there's no questions 
you 're doing it right . . . but if she feels you 're a bit iffy then she 'll 
say do you think you need to do your stretch twice, so it makes you 
think, and then if you correct yourself it’s noted because at least 
you then know. 

Sp&Rec NVQ Candidate 

Our primary form of assessment is through observation and 
questioning, if ... I’m not satisfied that the learner has met the 
standards, I will make reference to it in the feedback that I give 
to the learner, obviously I’ll try to be as positive and encouraging 
as I can; I’ll say that there’s been missed opportunities here, 
therefore I cannot use this piece of evidence as assessment. . . 

If they need additional support I will make every effort to make 
a weekly visit as opposed to a fortnightly visit for that learner, 
to give them the encouragement, to give them whatever training 
needs are necessary, and to encourage them to get through, 
and obviously I ’ll liaise with their managers. . . 
external assessor, Training Agency Sp&Rec NVQ 

In other workplace settings support can be observed in the way 
‘leading questions’ are asked by assessors of a ‘good lad’ to help 
him through observations of his workshop practice and compile 
his portfolio evidence: 

Our lads have got a matrix to follow and that’s been a bit of a job to 
get them to look at it and say well how many brake jobs have you 
got, how many clutch jobs have you got, how many engine jobs 
have you got. Well we don’t do engines. Well alright you do cooling 
don't you, you change a water pump or a fan belt which drives the 
water pump, or a timing belt. Oh yes we do that. Well that’s an 
engine job isn’t it... Oh we never thought of it like that... that’s how 
I feel the assessor’s job should be is to help them, guide them, 
show them what’s going on. 

MVE assessor 

In the example below, the interaction is more like a traditional 
‘teacher-pupil’ pedagogic encounter than a workplace 
assessment: 
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Fieldwork observation: 



Internal assessor 9 This is a service, yes? 
MVE trainee Yes. 



9 NB: although the term ‘internal 
assessor’ is used here, the assessor 
is not internal to the workplace, but 
internal to the Apprenticeship 
programme involving a local college 
and several workplaces. The internal 
assessor in this case is a college 
tutor who visits a number of 
workplaces as internal assessor. 



I A Replace two rear brake lights? 

Tr Just the rear brakes really. 

I A Did you find anything? 

Tr No, well it needed a fair bit of adjustment like. 
IA What did? 

Tr The shoes. 

IA Did they? 

Tr A fair bit of adjusting. 

IA What were the shoes like then? 



Tr Oh the shoes were alright. 

I A What were the drums like? 



Tr They had a bit of rust on them so we cleaned the drums out and 
cleaned the shoes out. . . 



IA I’m trying to think of something I haven’t asked you before. Yes, 
what causes the rust on a brake pipe? 

Tr Corrosion. 



I A So you get outside corrosion, yes, from the weather and then 
what about the inside corrosion? How does a brake pipe get rusted 
on the inside, which you can’t see? 

Tr The brake fluid gets warm. 

IA No. 



Tr It’s something in the brake fluid isn’t it? 

IA Yes, what causes rust? 

Tr Water. 

I A So if it’s rusty on the inside, what do we say the brake fluid does? 
Why do we have to change the brake fluid? If I say hydroscopic to 
you, I’m not swearing. Have you heard of it? 

Tr I’ve heard it now. 

IA Do you know what it means. Can you remember what it means? 

It absorbs moisture. So that’s why you have to change the fluid so 
that the brake pipes don 't become rusty on the inside. 

Tr / knew it was something in the fluid. 

IA Well now you know, don’t you. Don’t forget next time, will you? 



Such interactions beg questions of how such relationships are 
established and how a ‘good lad’ is identified in the first place. 
From the employer’s perspective, this seems to derive as much 
from the apprentice’s attitude to work as from their initial 
competence, and an identity is constructed between workplace 
supervisor, internal assessor and external verifier: 



His boss was putting a cylinder head back on a bus ready for a 
school run the following morning and he couldn’t manage it 
himself and this was 7pm at night, the lad’s already been working 
all day and he’d gone home. The boss rings up, can you come 
down and give me a hand. Yes, OK I’ll be there in 5 minutes. 

Comes down, works till 4am in the morning ... and he’s there 
again at 7.30am in the morning ready to see the bus out. You don’t 
get that with modern kids these days. His boss thinks he’s the 
best thing since sliced bread. 

MVE assessor 

They’d started a cam shaft on the . . . by-pass so they’d been out till 
one o 'clock in the morning bringing it back into the garage to sort it 
out. .. So he ’ll do that, but he won ’t stay here to learn about 
numbers because he doesn 't like numbers. . . You speak to him 
and it’s all ‘...errr...’ 

FE college tutor and MVE assessor 

Similarly in Social Care we observed assessors asking leading 
questions to help candidates articulate what they (supposedly) 
already know and can do: 

Fieldwork observation: 

Assessor Why is it important to explain these limits to clients - if they could 
speak, if they were old enough? 

Learner / don 't know. My mind’s gone blank. 

A So what if a child came home from school and they’re going on a 
trip in two days’ time and they want you to sign the form? 

L I’d say to him because I’m not your legal guardian, I’m not allowed 
to sign so we 'd have to try and get in touch with the parent or 
social worker for them to sign. I wouldn’t be stopping them from 
going on the trip. It’s not me that’s stopping them; it’s that I’m not 
allowed to sign. 

A So right, they know what their expectations are then. When you Ve 
got older children, Julie, it’s always important to explain everything 
to them. Not just saying I can’t sign it, that’s no good for the child, 
the child wants to know why you can ’t sign it. . . So let’s think, say 
for instance you get a child who 's not going to school at all ... but 
you managed to work with them and they start going to school and 
start going regular, why is it important that you sit down and talk to 
that child about what they’ve achieved by going? 

L Well it’s to help them further their education; get a better start 
in life. 

A How do you think that would make them feel about themselves? 

L / think they’d feel more secure and confident about themselves. 

A That’s what I’m looking for. 

L I finally got there! 
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Adult learning 

In Access courses and adult basic education settings, similar 
levels of support, coaching and practice were observed. 
Assignments were drafted, read, commented on and then 
re-submitted: 

I tend to lay out fairly clearly what I want. .. I've broken it down into 
four discrete sections and lay out exactly what they have to do for 
each and then they get handouts which will support them. 

Access tutor 

We try to keep it constructive but sometimes you have to point 
things out and you try and do it in a very supportive and gentle sort 
of way. Some people are not very good at taking negative stuff at 
all no matter how kindly you are. . . 

Access tutor 

They do give you a lot of feedback on your assignments. . . The first 
assignment in Psychology I got a [level] 2 and she went through it 
and she said if you define that a bit better than that and she gave 
me another week and I did it all and she gave me a [level] 3. 

Access student 

In basic skills programmes there are continual efforts made to 
relate literacy and numeracy tasks to relevant social and 
vocational activities, and render the adult basic skills Unit (ABSU) 
national curriculum into ‘small chunks’, but with the additional 
proviso that in college-based settings as much testing as possible 
was embedded in ordinary classroom activities: 

We all have small chunks for the students to cover and so they are 
doing activities in the classroom and they can self-assess. They 
can peer-assess as well. They are doing group work and they are 
doing listening as well, so they are assessing all these skills as we 
go along. You need to do that to show progress. 
adult basic skills tutor 

Right now I have been working through units, I am just working 
through the units each time. . . I tend to do them one by one, make 
sure they are OK, and get them out of the way. . . I think how it 
works is that you do one paper, and if you get it all right, then you 
can move on to the next one. . . It is the set-up to work through 
each unit and you take your time going through it, make sure it is 
correct, and you get to the end and do the final test. 
basic skills learner 

In this respect it is a moot point if some candidates even realised 
they were taking a test. A number of learners had progressed from 
one level to another without being able to remember that they 
had taken tests: 

You have to be very low key about it. . . You don 't want people to 
feel as if they’ve sat a test when they go out. 
adult basic skills tutor 



We try and make [testing] less formal because it’s in a classroom 
environment, they're not on individual desks in rows or anything 
like that. 

adult basic skills tutor 

The English teacher is quite clever, she 's given me a few tests 
without me [realising]. When I go on the computer she says ‘Well 
that’s Level 1 or Level 2’, so she says ‘You passed that. ’ I don’t 
know . . . which is kind of good because of the psychology of it. . . 
basic skills learner 

Observational evidence demonstrates this in action: 

Fieldwork observation 

The tutor sets each of the new students a task, Alice to write an 
autobiographical piece about herself and Beth to write a letter to 
her sister. When they have completed these, the tutor goes 
through the work with each individual, feeding back lots of 
positives about what the student can do. . . The tutor asks Alice if 
she has done an initial assessment, and she replies no. The tutor 
then says, ‘That’s part of one you’ve done already, actually. ’ She 
passes Alice a sheet of questions, and says casually, ‘Just do this 
for me, it’ll only take 15 to 20 minutes. ’ It is the rest of an initial 
assessment. Later she goes back to Alice and goes through the ... 
test. . . She feeds back to Alice that she appears to be at the 
top end of Entry level 3 - ‘nearly at Level 1 GCSE’. The student 
expresses her surprise and pleasure, and at the end of the 
session thanks the tutor for the way she has created a supportive 
atmosphere in the lesson... 

Significant levels of tutor ‘hidden work’ were also observed in this 
sub-sector as in others: 

If you've got problems you know who to talk to. You haven’t got to 
go hunting for somebody. . . I think [tutor] R feels like the mother of 
the group. She 's the mother hen that goes round and worries 
about everybody. . . If you need help or you need extra time, you go 
and talk to R and it's sorted. 

Access student 

Last year [named tutor] had them from [village] and it’s quite a way 
out, I mean it’s right up in the hills really. And they don’t have cars 
because it’s a poorer area. . . And she brought them down in her 
car. She went up, collected them all, brought them all back. They 
did their exams and she took them back again. Because you do 
worry about them. Well, I worry about them. I know it’s stupid. 
basic skills tutor 
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Quality and equity in learner support 

None of this support, even coaching, is necessarily inappropriate 
or unfair in and of itself. Such practices are at the heart of 
professional judgments about the performance/competence 
interface which assessors must make. But they raise issues of 
equity if they are not pursued uniformly, and the questionnaire 
data suggest that there can be wide variations in the frequency 
and length of assessor visits. Thus for example, while the most 
frequently reported timing of assessor visits among the NVQ 
takers was 1-2 hours every 4-6 weeks, one reported that they 
saw their assessor once a week for 2-3 hours, while four reported 
that they saw their assessors only every 3 months or less, and 
for one hour or less (Appendix 3). There is also evidence from 
the case records that such discrepancies may derive from the 
different cultures that develop in colleges and training agencies, 
with college tutors not only seeing apprentices in college, but also 
often visiting them in their workplaces to conduct assessments, 
and thus developing a much closer pedagogical relationship than 
that of training agency staff, who simply travel around workplaces 
assessing full time: 

Nobody’s looking at my portfolio or nothing . . . [the named training 
agency] don 't even come to work any more ... [he] came when 
I first started about 4 months ago and ...I haven 't seen him since. 
I tried ringing him and he 's just said, ‘Oh I ’ll get back to you or ring 
up a bit later, ’ but nobody ever rings back or anything. . . 

MVE trainee 

Equally important is the attitude of internal and external 
assessors and the quality or vitality of the relationships they 
develop with candidates: 

Most of the feedback you get is that [candidates ] don 't enjoy it 
[ie the process of assessment], it’s tedious, it’s cumbersome, 
it takes time out of your real job [but] there are people who enjoy it 
and benefit from it and i guess a lot of that is down to the quality 
of the assessment, I think cos if you’ve got a good assessor who’s 
enthusing you, it will have a different effect on you than if you’ve 
got someone you’re not getting on with. It’s like anything: 
whoever’s involved in it, is crucial. 
internal verifier, Social Care NVQ 

Some regulation of and minimum recommendations for such visits 
would seem to be appropriate, as would be clear instructions to 
assessors that the quality of the assessment interaction is likely 
to be as important to ‘fair’ and motivating assessment as 
the accuracy of the observations and record. Stasz etal. (2004) 
and Fuller & Unwin (2003) similarly note the importance of the 
vitality and range of workplace relationships for the quality of 
trainee learning. 



Helping candidates ‘complete the paperwork’ is also an important 
element of the assessor’s role in assessment interactions: 

The work itself isn’t difficult because that’s what I do every day in 
my job; it’s the entire assessment process, that’s the hard bit- 
getting it all on paper ... [it’s] ... time consuming, complicated with 
all the codes and all the cross-referencing and terminology. . . 

Social Care learner/candidate 

Yes, it’s a learning process for everybody. When the assessors 
know the language but the candidates don 't, then the assessors 
can do all this cross-referencing for candidates... 

Social Care manager/assessor 

In garages it appeared to be a very common practice for MVE 
apprentices simply to keep their garage ‘job-sheets’ up to date 
and filled in with brief descriptions of the jobs undertaken 
and completed. The assessor then ‘deconstructs’ this basic 
information into the relevant ‘competences’ and maps and 
transfers the detail into the apprentice’s evidence portfolio: 

MVE trainee In the workshop itself we have to do a job-sheet for everything 
we do, whether it be change a tyre or anything else. Basically 
whenever there’s a defect put on the screen we have to do it 
and sign it off and then we have to keep it in the files for however 
long it is needed. . . [The assessor] said that you just write down 
every single one and he ’ll look at them. 

Interviewer So eventually he ’ll come in and look through them all and pick 
out the relevant ones?’ 

MVE trainee Yes, that’s right. 

Such practices also accord with the findings of other recent 
studies of portfolio completion in the workplace, which indicate 
that younger workers do not usually take responsibility for 
portfolio completion (Kodz et al. 1998, Tolley et al. 2003). Indeed 
Fuller & Unwin (2003), in their study of apprenticeship in various 
sectors of the steel industry, note, almost in passing, that: 

Responsibility for recording the apprentices’ progress towards the 
achievements of the qualifications was taken by the external 
training provider at the regular review sessions. An important part 
of his job was to help apprentices identify how the day-to-day task 
in which they were engaged could be used to generate evidence 
that they were meeting the competence standards codified 
in the NVQ... 

(Fuller & Unwin, 2003, p422) 
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In all of this it is also important to note the different definitions 
of ‘fairness’ which tutors and assessors work with, which in turn 
relate back to the introductory discussion of different 
constructions of achievement, viz: 

i helping the learner to achieve the grade or the qualification they 
‘deserve’ and, especially in NVQ, helping a ‘good lad’ get through 
who can do the job; or helping a care worker who ‘can do it 
anyway’ get the evidence to prove it. This in turn begs questions 
of how the assessors ‘know’ they ‘can do it’. There are issues here 
to do with the validity and submissability of informal assessor 
observations and how candidates’ identities are constructed over 
time - how do assessors ‘know’ certain learners ‘deserve it’? 

ii testing the learner under ‘exam’ conditions - precisely without the 
interference of the tutor. NB: this construction is not restricted 
only to certain tutors. It is also apparent among learners, 
including those who avoid external testing if possible but 
nevertheless see the argument for it and even concede the status 
that flows from it. For example, as we have seen, some Access 
course learners were unhappy that their peers had ‘so many 
chances’ to submit acceptable work; similarly some adult English 
as a Second Language (ESOL) learners preferred formality in 
teaching and testing. 

Transparency promotes instrumentalism 

A corollary of the level of support provided is that not only is it 
both demanded and expected by learners, it is also expected to 
be very specifically focused on achieving the qualification: 

Sometimes we have to do other tasks but the bottom line is 
‘Is it relevant’... [There’s] no time to do irrelevant stuff. 

AVCE student 

In Economics, he just loves his Economics - he loves it too much! 
He knows his stuff, don’t get me wrong, but he watches all these 
programmes and records them for us and we have to watch them, 
even if there’s no exam question for them. 

A- level student 

In turn the instrumentalism of learners both drives and validates 
the level of tutor support. Similarly, institutions themselves are 
also target-oriented and instrumentally driven: 

We certainly have a target system . . . where every month we have 
pre-determined targets as to who ’s got to be complete, who ’s due 
in for an interim verification, who’s due in for reviews, etc ... it 
comes down to the issue of: is it feasible to get a learner through a 
Foundation or Advanced Modern Apprenticeship programme within 
the time-frame... I want everyone to achieve the best possible for 
them. If they need additional support I will make every effort to 
make a weekly visit as opposed to a fortnightly visit. 
assessor, training agency, Sp&Rec NVQ 



Thus learners seek and expect details of assessment 
specifications, evidence requirements and so forth. They want 
support and appreciate it when they get it; but their 
instrumentalism reinforces tutor moves to focus on grade criteria, 
the elucidation of evidence, etc. As a consequence, assignments 
and portfolios from some institutions can often look very similar - 
in structure, format, types of evidence included, etc - so it seems 
that some institutions are becoming very adept at ‘coaching’ 
cohorts through assignment completion; exam cramming by 
another means. To reiterate, this is what we term assessment as 
learning, with the assessment process ensconced at the heart 
of the learning experience and defining every key aspect of the 
learning experience. 

This finding of extensive support, manifest across all sub-sectors 
of the LSS through coaching, practice, drafting and the elicitation 
of evidence, coupled with an associated learner instrumentalism, 
seems to derive in large part from the move towards transparency 
in assessment processes and criteria. The more clearly 
requirements are stated, the easier it would appear for them 
to be pursued and accomplished. This can in turn lead to a very 
boring ‘paperchase’ for tutors and learners alike, especially in 
college-based NVQs, which lack the advantage of being located 
in a working environment where the paperchase does at least 
relate to authentic working activities: 

The trouble with the NVQ is that it breaks everything down into 
tiny bits, and you can spend ages just trying to find some direct 
evidence that will allow you to tick a box. It is difficult to come 
up with a way of running the course so that they get a lot out of it, 
but don ’t have to sit for hours in a classroom ticking boxes . . . 

They could divide down making a cup of tea into 10 criteria . . . 
and did you wring out the tea bag afterwards? You didn’t? Oh well, 
we can 't tick that box then. . . 
college-based NVQ tutor 

Equally, however, the imperative to compliance and the ‘expulsion 
of failure’ (except with respect to non-completion) begs questions 
about what should now constitute a legitimate learning challenge 
in the context of post-compulsory education and training. The 
danger is that as the number of enrolments and awards achieved 
increase, the underlying purpose of such expansion - increasing 
the numbers and improving the standards of vocationally-qualified 
workers - may be compromised, with candidates being ‘coached 
to ‘comply’, rather than ‘learn’. 
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Two cautionary riders have to be added at this point, however. 
First, we are dealing with learners who are in the system, rather 
than outside it, and hence will have a propensity to continue to 
comply and achieve, albeit while perhaps avoiding tasks or 
activities perceived as being too difficult. Second, we have 
encountered evidence of more intrinsic orientations and rewards, 
particularly in the adult education sector, with respect to the 
development of self-confidence, as noted previously. In training 
contexts, the development of practical competence following 
school-based academic failure and/or disinterest has also 
brought a sense of achievement. By their very nature, however, 
such achievements are difficult for learners to identify and 
articulate in more than a cursory manner. When asked in the 
questionnaire to state how they knew they were making progress, 
learners’ answers varied from ‘because I’m passing the tests’ and 
‘because I’m getting more knowledge’ through ‘holding myjob 
down at work’ to ‘because I just re-fitted the brakes on my car’. 
Clearly there is a sense of personal achievement linked to 
developing competence in these latter responses, but such 
responses are unusual and hard to ‘call forth’ in the context of 
assessment tasks and events where achievement and progress 
are normally interpreted much more narrowly in terms of 
awards and certification. 




Linked to transparency and instrumentalism are perceptions of 
progression and routes to success. 

Col lege -based factors 

The main route to learner success, certainly in college-based 
work, is criteria compliance. This is supported by the provision 
of detailed guidance and feedback, and multiple opportunities 
to achieve. 

A key problem with this, however, is that it may foster a lack 
of ambition - aiming low, working in the ‘comfort zone’ of 
‘pass/merit’ rather than ‘distinction’, orCs and Ds rather than 
aiming for As and Bs at A-level. Thus learners aim to achieve the 
minimum ‘necessary’ for (already decided) progression, which 
is relatively easy to identify and accomplish within a ‘widening 
participation’ agenda and the expansion of higher education, 
rather than achieve the maximum possible which might be aspired 
to and which might, crucially, open up new tracks/pathways. 

I’m heading for a C and that’s OK. 

AVCE student 

I'd be happy to get a C but I know I could do better. It works on 
points so it’s easier to set that target. 

AVCE student 

We get students thinking ‘Ok, we can just re-do it, ' so there ’s no 
sense of urgency or effort. . . Last year two girls were confident of 
getting a B and so they put less effort in so they could concentrate 
on other subjects but they could have got an A. 

A-level BS tutor 

Thus for example the decision of one college in the study to return 
to BTEC National Diplomas ‘to do all the things these kids love ... 
to move away from all this written assessment’ also meant that 
vocational students were ‘more supported, it’s to do with comfort 
zones - a more protected environment’. The problem is that 
students perhaps become over-protected, and the very perception 
of the provision made for them may restrict their horizons rather 
than broaden them. 

The possibility of attempting to achieve beyond expectations 
and thus open up the opportunity to do something different was 
virtually never articulated. Where it was, it tended to be in the 
context of mature learners aspiring to ‘better’ themselves - 
gaining qualifications to move into more professional employment 
roles. Complicating this issue even further in some programmes 
at college level is the criteria for ‘distinction’ which, as we have 
seen, demand that very little tutor help is given to the learner. 

Thus good formative feedback and encouragement to do better 
might, by bureaucratic definition, mean that college students who 
do become motivated to achieve more are denied the ‘distinction’ 
grade that such pedagogy has encouraged them to pursue. This 
tension was acknowledged by a senior awarding body manager 
we interviewed, though he also reflected that it was not an issue 
his organisation had previously recognised or addressed. 
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Work-based routes 



10 It is also important to note that, 
even with respect to professions 
with graduate entry, whole 
research projects have been devoted 
to, and typologies developed for, 
investigating and representing 
competence in action. It is actually 
very difficult for individuals to 
articulate what they know and 
how they have come to know it 
(Eraut 2000). 



A key indicator of success is also the development of practical 
competence(s) - becoming, for example, a skilled mechanic, and 
a more confident operative in the workplace. However, this doesn’t 
feature immediately in learner accounts. The explanation for this 
may be that the development of such practical competences are 
thought of as somehow taken-for-granted, with skills being seen 
as bound to develop with instruction and practice; in contrast, 
‘passing the test’ and accumulating ‘the paperwork’ is seen as 
difficult, and hence a notable achievement. Articulating the 
developing nature of practical competence is also difficult to do 
and, as we have seen, tended to be illustrated by brief indicators 
such as ‘holding down my job alright’ and ‘mending the brakes 
on my car’. Supervisor and assessor support for transforming 
observed activities into written descriptions of practical 
competences is certainly important for the achievement of 
awards (cf. Sections 4.4 and 4.9). Thus the data indicate that, just 
as trainees could not easily articulate competence development 
to the research team, nor do they do this in the contexts of their 
awards in any case; supervisors and assessors act as 
intermediaries in this process. 10 

A further feature of developing and displaying competence, 
however, is having the opportunity to do so. Much of the data 
relating to problems with workplace training and assessment 
pointed to poor workplace support and lack of opportunity for 
candidates to demonstrate competence - with apprentices being 
given inappropriate jobs (eg helping with vehicle recovery rather 
than actually working on repairs) or employers not providing the 
full range of equipment or workplace activities for candidates 
to practice and complete tasks: 



I work in an old-fashioned garage where we’ve only got basic tools, 
I can 't take it [NVQ Level 3] without having diagnosis equipment. 
MVE trainee 



Essentially all the facilities which we ... deal with sign a learner 
and training agreement which basically says for every learner 
they’ll be given sufficient time and support to achieve the units of 
competence required of them. However, in many of the facilities 
that we deal with . . . the limitations on time within this environment 
tend to prevent that from occurring . . . the demands placed on 
[ learners ] by their employer are such that they don 't get sufficient 
time and support during work hours to collate and collect their 
evidence for their NVQs. 
assessor Sp&Rec NVQ 

I’m not actually being allowed any time at work at the moment to 
do it. . . There 's an agreement that we should be allowed a certain 
amount of time ... [but] there’s not much staff here at the 
moment. . . We Ve been put in a locked room for an hour and a half 
to scrub toilets, and it’s not really my job, to be honest but . . . 
when staff leave, you’ve all got to muck in haven’t you... 

Sp&Rec NVQ Level 3 learner 



A lot of the time the boss will say come into work instead of going 
into college because we’re busy... It was just happening now and 
then and then it started being nearly every week so I just said 
I’m leaving because I need to go to college and he lets me go now 
every week but the other apprentice, he’s not allowed to come, he 
never goes to college... He’s missed most of the year so I think 
he’ll have to start again next year. 

MVE trainee 

The learning environment in workplaces can also vary enormously 
with some apprentices becoming very isolated: 

Fieldwork observation: 

It is a fairly rundown-looking building. The workshop is like 
a barn ... and the office is in a mobile caravan. The workshop 
is cold and dirty. The proprietor is eating fish and chips out of 
newspaper in the office and he says the apprentice is also having 
his lunch. We find the apprentice in a different office, a small 
shed-like construction, eating his lunch alone. When they have 
finished [MVE assessor] talks to the apprentice about his work. 
The trainee has been working on changing a complete engine... 

He has been doing most of the work alone. [MVEA] fills in the job 
card as he talks. The trainee doesn 't say much while [MVEA ] 
spends almost 30 minutes explaining technical details to him 
about engines and asking him questions. Eventually, the trainee 
begins to ask a few questions. [MVEA] tells him he is very lucky to 
do such interesting work and most lads weren’t doing things like 
changing engines. [MVEA] explained [to observer] that he spent 
more time with this trainee because he is quite isolated and very 
difficult to talk to. 

And just as a learner’s opportunities may be limited by 
circumstance, so may the employer’s: 

The other issue ... is: are the employers occupationally 
competent? And are they fully conversant with the standards that 
we need to address? Managers or the supervisors at the facility 
may well be conversant with the standard operating procedures of 
their individual facilities, they are not fully conversant with the 
national standards as stipulated in the technical specifications 
and the standards for the NVQ, and therefore ... witness 
testimonies don’t tend to prove a valid piece of evidence... 
assessor Sp&Rec NVQ 

Thus an external assessor’s job can also be to inspect and 
educate the employer as much as the trainee: 

Observe how he 's being trained by the employer. . . Sometimes you 
have to tell the employers off, you're expecting too much of this 
lad, have you shown him what to do? Have you taken a bit of time 
and explained how to do this? How do you expect this poor lad to 
do what you want him to do if you haven’t got the time and 
patience to show him how to do it? 

MVE assessor 



LSRC research report Section 4 page 60/61 



Even when candidates receive direct instruction in the workplace 
from supervisors and/or visiting training agency staff, the quality 
can also vary: 

Some of the lessons we had were appalling and I just used to sit 
and think how insulting it was. . . I remember once talking about 
hydrating a client and she said, ‘How do you hydrate a client?’ and 
started talking about drips and I’m like well would you not offer 
him a drink first. 

candidate completing Social Care Level 3 

At the same time, properly presented ‘underpinning knowledge’ 
can help to motivate as well as inform employees: 

They get a question which says: ‘Pressure care is important, do 
a reflective account on what measures you undertake to prevent 
pressure sores’. Now it’s a very big subject, it’s not just about 
them lying in a bed or sitting in a chair, you’ve got to know a whole 
lot about nutrition, about the blood supply, about the nerves 
supply. Now some people would argue they don 't need to know 
all that . . . but ...I expect them to understand that if the bum is 
lying on the bed like that, hour on hour, the blood which is taking 
food to the skin ... cannot get there, so the skin isn’t getting the 
food it needs. That's how I explain it to them . . . before we started 
on NVQ . . . they didn 't understand why they should mobilise . . . 
before we came they put a pad on them when they got them 
up in the morning and they sat them there and they gave them 
their dinner there and they gave them their tea there and at 
8 o 'clock they took them to bed and that’s what they did and 
that's why the place was going to be closed and I’m going on 
about mobilising, walking, going to the toilet, pressure care ... 
the underpinning knowledge... 
internal assessor Social Care NVQ 

Similar issues are discussed in Section 4.8 below regardingthe 
validity, reliability and equity of assessments. Such findings also 
accord with Fuller and Unwin’s (2003) observations on ‘expansive- 
restrictive work environments’ with respect to the opportunities 
for learning offered to apprentices by different workplaces. They 
argue that ‘expansive’ participation allows access to multiple 
communities of practice, as well as to on-the-job and off-the-job 
learning opportunities, and to knowledge-based as well as 
practical qualifications. Expansive work environments offer 
mentor-rich communities that support learners, and they 
encourage experienced colleagues to welcome the new knowledge 
that novices themselves bring into the environment. 

Adult learning 

The development of self-confidence also featured in the data, and 
the interaction of confidence with achievement, especially in the 
context of adult and community education. Facilitating success in 
this sector involved a very delicate pedagogical balancing act 
between gentle support and encouragement, coupled with very 
small incremental steps to achievement, eventually leading to 
more substantial challenges and achievements. Interestingly 
enough, e-assessment of basic skills seemed particularly helpful 
in this context because of the capacity for learners to take tests 
virtually ‘on demand’ (when ready) and receive immediate 
feedback. Section 4.6 develops this theme. 




E-assessment was encountered in the context of MVE 
Underpinning Knowledge tests, and Adult basic skills testing. 

It was also the focus of a small extension project funded 
separately by Ufi with respect to the use of e-assessment in NVQs 
and Learndirect centres. Evidence from the extension project 
is drawn on here as appropriate, and the project is also the 
subject of a short, separate report (Jarvis & Torrance 2005). 

E-assessment is becoming more widespread across the LSS 
but comprises two distinct elements - e-testing, ie the online 
provision of multiple-choice tests, and e-portfolios, these being 
online provision for the recording of assessments and other 
forms of evidence. Sometimes respondents use the term 
‘e-assessment’ when they only have experience of one or other of 
these elements and take the term to mean only that activity. 

The use of e-testing is much more widespread than that of 
e-portfolios, which are still largely in the development and piloting 
stage, and the benefits of e-testing are much more apparent in 
the data than those of e-portfolios. In principle, e-testing is 
available online 24 hours a day, 365 days a year. In reality, 
e-testing is much more likely to be accessible on screen via 
a local area network. Tests are downloaded through an online 
administration interface. 

Institutional benefits of e-testing revolve around efficiency of 
administration and motivation of candidates. Candidate 
programme registration can be easily integrated with test 
submissions, and substantial fee savings can accrue for colleges 
by only entering candidates when they are ready. One college 
in the study had previously wasted up to £18,000 pa on test fees 
for candidates who had to be entered well in advance of sitting 
paper-and-pencil tests, but didn’t appear on the day: 

We put 7500 students through the end tests last year, which is 
a hell of a lot. And of them, 48% failed to turn up. We paid for that. 
So that is 3000 students at least who failed to turn up for an exam 
that cost us £5-6 plus the invigilators. 
adult basic skills tutor 

With online access, the tests are simply not ‘unlocked’ on the 
computer unless and until the candidate is available and the fee 
is not incurred until the test is taken. However, there can also 
be problems of access to hardware and software if a dedicated 
e-assessment suite is not part of college provision, and 
sometimes even when it is: 

We haven 't got a dedicated room here. .. We have a teaching 
computer room which is laid out and we 're using it for online 
testing and every time we come to do testing somebody’s either 
loaded some other packages on the computer or changed the 
profile so we can’t do it and I am running round like a headless 
chicken trying to get the IT to sort it out. 

FE college MVE head of department 
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The trickiest thing is when you come to download your scripts 
and they’re not there on the morning. . . Something gone wrong. 

A hitch with the Board, they haven’t uploaded their system or 
our exams haven 't uploaded ...or whatever . . . There is a helpline 
for City & Guilds, and they are very good, if you ring them they’ll 
tell you if it’s their mistake, or they’ll say you've made a mistake 
on such a thing. . . 
test centre administrator 

The combination of multiple-choice format and electronic 
processing means results are available almost immediately. 
Learners and tutors alike comment on the educational benefits of 
taking tests when ready, and the immediate feedback of results. A 
pass allows learners to move on quickly rather than waiting weeks 
for a result. Equally, however, if a fail is received learner and tutor 
are usually still together in college and able to sit down and review 
the situation. Re-takes can then be scheduled to suit the learner: 

You know within half an hour to an hour. .. It’s a provisional grade 
but then they go away, and they know, and they’re happy. Even 
those that haven’t passed don’t have to wait to be told ... [it’s] ... 
horrible to wait to be told you’ve failed something. So it means 
it’s done and dusted, and it’s not as painful for them . . . you 're 
there on the spot to say, ‘Never mind, do it again, ’ instead of them 
stewing and winding themselves up about it. 
adult basic skills tutor 

I am happy with [online testing], because I think it is good for the 
students’ morale. They get an immediate response and that is 
good for the students. They are not waiting for the results and they 
know straight away whether they need to re-sit or not. I think this is 
a big advantage for all students, in particular mature students, 
who tend to worry a lot about how they are doing. 

Adult basic skills tutor 

The multiple-choice format might suggest too much formality in 
the test situation, given some of the views previously reported 
about the anxieties which can be generated by testing, but in fact 
both tutors and learners liked the relative informality of e-testing 
when compared to the formality of the traditional ‘exam room’: 

I enjoy it on the computer. [The test] is easy, because if I make a 
mistake, I correct it ... It 's like a game, it 's not difficult. When I do 
the computer test, it’s like a game [laughs]. It's no pressure for 
us, because if I write with pen and paper, we think it’s something 
hard. I think, ‘Oh, is it right or wrong?' I think like confused, but 
computers are not confused, I think it’s like a game. 
basic skills learner 

You are not sat in an old school sports hall or cold classroom in 
uniform rows and some sort of writing against the clock. You’re in 
a nice assessment centre, informal-looking, it’s warm, it’s 
modern-looking. . . 
adult basic skills tutor 

They’ve actually used [computers] before they come in to do the 
online testing and are quite used to them and don ’t see it too 
much a problem . . . they have done all the practice testing ... a lot 
of work on the computers for the course and practice tests on 
the computers... 
college test centre manager 



E-testing also allows students to have multi-modal access to the 
test items, and this can facilitate their response: 

On the PC version, it’s clearer more, you get pictures, you get all 
this sort of colour and everything, and on the paper you don’t get 
nothing like that. You get all the visual everything [on computer]. 
Paper work is like . . . things written down in black and white, but 
when you go on the PC, it’s explained in colour, the same kind of 
things, but it’s in more depth. And that way makes it easier, 
a lot easier. 
basic skills learner 

[On the computer] it’s more colourful. It stimulates your brain 

a bit more. 

basic skills learner 

A number of learners also liked the multiple-choice format and 
‘point-and-click’ mouse controlled technology, because it limited 
the need for extended writing and (they thought) was open to 
guessing: 

It’s good ... [there’s] ... less time used on computer than 
in writing. We use our eyes, very quickly, time-saving. 

It’s painful, writing! 
basic skills learner 

I did [the initial assessments] on paper and on computer. I’d rather 
do it on computer, that’s easy to use . . . cos I just have to look at 
the question and then just click the answer. . . On the paper, I need 
to write everything out. It’s a bit hard. 
basic skills learner 

Tutors, while generally welcoming e-testing, disliked multiple- 
choice formats for exactly the same reasons that the student 
liked them - tutors thought that they were an invalid way to test 
literacy - see below. 

Students also appreciated the very fact that they had to become 
familiar with computers in order to sit the tests. Learning to use 
a computer was an additional ‘embedded’ skill that they enjoyed 
learning. However, this is very much associated with the 
‘point-and-click’ technology. Extensive compilation and 
manipulation of e-portfolios is likely to prove far more challenging 
for new computer users. Some students were resistant to 
word-processing in any case, and therefore to e-testing, and 
this led most tutors to favour a continuing dual system wherein 
learners would have the option of paper-and-pencil tests: 

I don 't like using the computer. I don 't know why. I don 't like it. 
Always I told myself I can learn the computer at any time, the 
important thing now is learning English. 
basic skills learner 

If I write it down on paper, I know which spelling is right and which 
is not, remembering. That's why it's best writing down on paper - 
not computer. Some people are lazy, they do it on computer, 
problem solved, no brainwork. 
basic skills learner 

I wouldn’t know how to turn a computer on. It’s hard enough with 
a pen trying to spell something, without having to look for every 
single letter. Whoever designed [keyboards] and put them letters 
in that order wants shooting at dawn. It’s ridiculous. 
basic skills learner 
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Laptop technology, including wireless access, also allows such 
testing in a wide variety of workplaces, and this seemed to be an 
important selling-point in persuading management and unions to 
become involved in workplace Basic Skills programmes; the tutors 
and the test can ‘come to them’, rather than the learners having to 
go to the college. At the same time, however, small and/or 
isolated workplaces such as care homes may well find it difficult 
to offer or take up such opportunities, lacking the hardware, the 
wireless and/or broadband connectivity and/or the IT skills. 

The negatives of online testing, especially with respect to literacy, 
largely derive from the fact that the tests are narrow multiple- 
choice tests of reading and recall - using a mouse to tick boxes 
via ‘point-and-click’ technology. There is no testing of extended 
writing or application of knowledge. However, this is not a specific 
feature of e-testing per se; it is also a feature of the paper-and- 
pencil version of the tests: 

That’s true of the paper-based as well ... it's all multiple-choice. 
You could pass this exam and not be able to write a piece of 
continuous text. . . But having said that, this is going to sound awful 
to say, but I’ve got two students who are dyslexic, and it’s working 
in their favour, because they wouldn't pass a Level 1 exam if they 
had to produce some free writing, to be honest... 
adult basic skills tutor 

You get students who have very good Level 2 results, and when 
they come to written work, they can 't construct a sentence. . . 
adult basic skills tutor 

I can foresee problems where you are issuing a student with a 
Level 2 literacy qualification that they go out into industry and we 
say this is an A-C equivalency at GCSE and they’re struggling to 
write coherent sentences. 
adult basic skills tutor 

Tutors teach and assess extended writing through classwork and 
the compilation of a portfolio of evidence. But the portfolio is not 
considered by policy-makers to be ‘objective’ enough for basic 
skills accreditation at higher levels. Thus basic skills Entry Levels 
1, 2 and 3 (the most introductory or elementary levels, equivalent 
to primary school and early secondary school standards of 
literacy and numeracy, and currently accredited by NOCN in the 
colleges studied) require both a portfolio and an end test; but 
basic skills Levels 1 and 2, which also correspond with key skills 
Levels 1 and 2 taken by work-based trainees, are only tested by 
multiple-choice tests. These are accredited by City & Guilds in the 
colleges studied. The awarding bodies acknowledge problems of 
validity with multiple-choice testing, but state that they are 
operating under direct instructions from the QCA to employ this 
multiple-choice format for external testing of key skills and basic 
skills. It is ironic that that the very body established to oversee 
quality assurance in the assessment industry should insist 
on the use of such an invalid measure for the assessment of 
communication skills. 



In one instance, the elements that made the whole enterprise very 
efficient - setting up a dedicated assessment centre with properly 
equipped computers and an administrator always available 
to invigilate, etc - brought criticism from the Inspectorate about 
lack of ‘embedding’ of Basic Skills teaching and assessment 
in vocational settings. This was not wholly an issue to do with 
e-testing; it also reflected the college’s commitment (as the 
college saw it) to providing properly trained basic skills tutors 
to teach in vocational areas: 

Vocational staff haven’t necessarily got the skills to teach basic 
skills, they’ve got an awareness of it, they can support the basic 
skills, they do that in some groups . . . but to actually teach 
the basic skills to get people up a level which you need to do, 
you need someone who knows what they’re doing in basic skills. 

FE college programme manager 

However, the problem betrays an interesting ambiguity about 
whether or not it is more effective and efficient to teach Basic 
Skills completely embedded in vocational areas or not, an issue 
we will return to below in Section 4.7. 

E-portfolios were claimed to bring efficiency to evidence 
management for assessors and verifiers - they could deal with far 
larger numbers of candidates - but we have found little evidence 
of extensive practice benefiting learners. The evidence we have 
suggests that most claims for e-portfolios are just that: claims for 
how they might be able to be used in the future, rather than what 
they are actually used for now. 

E-portfolios are designed to be used as huge e-archives so that 
candidates for NVQs can add evidence to their portfolio over time, 
cross-referencing pieces of evidence to the standards or 
assessment criteria. Indeed, the software is intended to do much 
of the cross-referencing for them, as long as it is correctly inputted 
and indexed to the relevant national standards. Thus, in principle, 
instead of compiling very large paper portfolios and trying to work 
out where particular pieces of evidence ‘fit’, progress can be 
shown much more easily, even to the extent of having bar charts 
produced of completed elements and profile components so that 
candidates are encouraged by seeing such incremental progress. 
Evidence can be captured on video or audio and stored digitally 
in the e-portfolio, and candidates may also produce evidence 
which can be used at a higher level, and this can be stored 
appropriately. However, there is not currently a single technical 
standard for e-portfolios, and the platforms they use are all 
different. This could cause difficulties if a candidate leaves an 
employer or training agency to go elsewhere. The information 
could be transferred to a CD-ROM and then re-entered on to 
another system, but if candidates wanted to transfer a record 
of progress across into another package they would probably have 
to start again in terms of cross-referencing the evidence 
of achievement. 
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The most significant potential problems with e-portfolios, 
however, concern the level of IT skills which would have to be 
employed to use them effectively, and the implication from 
training managers and assessors that face-to-face visits to 
candidates would take place less often as each assessor would 
be dealing with much larger numbers of candidates. The 
implication that assessors might physically visit candidates less 
often could potentially undermine much of the ‘pedagogy of 
assessment’ that helps supports learners through the ‘evidence 
elicitation’ noted in Section 4.4, above. As we have seen, the 
quality of candidates’ relationships with their assessors is a very 
important element in the production of achievement. We have 
also seen that, in many cases, especially with younger workers on 
apprenticeships, the assessor actively interprets trainees’ tasks 
and job-sheets in order to record the NVQ competences from 
them for the trainee, rather than the trainee doing it themselves. 
E-portfolios probably could not be similarly completed at a 
distance from the workplace. Thus use of e-portfolios could raise 
issues of equity with respect to time spent with assessors and 
even jeopardise completion of portfolios and awards if most 
interaction was expected to be online. Guidelines on the use of 
e-portfolios, including the maximum numbers of candidates that 
can be taken by assessors and external verifiers, and the 
necessity for the continuation of site visits, would be appropriate 
before their use becomes too widespread and driven by concerns 
about efficiency rather than pedagogy - see also 4.4, above, 
regarding the frequency of assessor visits. 




Having discussed e-testing of basic skills and underpinning 
knowledge, and noted its popularity with learners, it is also 
important to report that key skills tests were universally loathed - 
by learners and tutors alike. 

There are significant differences in the way that basic skills tests 
and key skills tests are perceived by learners. Basic skills start 
with three ‘Entry levels’ which are below ‘Level 1’ and remain 
separate from key skills. However, basic skills levels and tests 
and key skills levels and tests are exactly the same for Levels 1 
and 2 (ie up to GCSE-equivalent level) and are often conflated in 
discussion. Nevertheless, they carry very different connotations 
for different categories of learners. There are very clearly 
perceived differences between the following three test areas: 

■ basic skills tests for adults 

■ ‘underpinning knowledge’ tests for other (often younger) 
learners, including apprentices, which test substantive 
underpinning knowledge of specific vocational areas, usually 
in tandem with NVQs 

■ key skills tests which can probably best be described as 
‘basic skills’ tests for 16-19 year olds, including apprentices. 



11 Formally, basic skills tests assess 
literacy, numeracy and ICT; Key Skills 
tests assess communication skills, 
application of number and ICT. 



Basic skills tests for adults are seen as an inevitable and in some 
respects helpful element of the modernisation and rationalisation 
of ABS teaching. They are increasingly accepted by tutors and 
learners alike as a prerequisite for progression to further study, 
including, if necessary, Access to HE programmes. Underpinning 
knowledge tests, including those for technical ‘progression 
awards’, are accepted by apprentices as relevant to the job and 
the NVQ. Key skills tests - tests of supposedly generic, 
transferable competencies in communication, application of 
number and ICT - are not so accepted. 11 Young trainees with 
relatively poor prior academic qualifications dislike key skills tests 
because they remind them of school failure; in contrast, relatively 
well-qualified trainees (eg with 4, 5 or more GCSEs at A*-C) 
regard them as an ‘insult to their intelligence’. Apprentices with 
good grades at GCSE have initial exemption from key skills tests, 
but only for three years. After this their GCSEs are regarded as 
‘out-of-date currency’, and candidates are required to take the 
tests to complete their apprenticeship. Given that this is most 
likely to happen to apprentices on Level 3 programmes which take 
longer to complete, but who are likely to be the best qualified 
entrants in the first place, it seems both ironic and perverse: 



It’s very difficult. I’ve got lads that are coming on these 
block-release courses who Ve got 3 or 4 O-levels at good grades 
and I’m having to tell them because they won’t achieve their 
NVQ for three years they’ll have to do their key skills because 
the currency of those qualifications will not generate the evidence 
to give them a bye. 

FE college MVE head of department 
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12 See also Hodgson & Spours 2002, 
Savory, Hodgson & Spours 2003, for 
similar findings regarding the 
introduction of Curriculum 2000; 
Savory et al. also note that key skills 
were required for the award of 
GNVQs, but are no longer required for 
the award of AVCEs. 



This situation does not pertain to any other uses of GCSEs; they 
are not regarded as ‘out-of-date’ for employer selection purposes, 
or university entry, if these take place more than three years 
after the examinations have been taken. Similarly, key skills tests 
are meant to feature in AVCE and A-level programmes but do not - 
schools do not have to offer key skills, and they are not required 
for receipt of the awards, or for progression to HE, so they are 
largely ignored . 12 In colleges, however, provision of key skills 
teaching and testing is linked to funding so that colleges are 
in effect obliged to offer them. Key skills are also required for 
the successful completion of Foundation and Advanced 
Apprenticeships (FAs and AAs), though many such trainees do 
not take them, or do so with very negative attitudes towards them. 
Non-completion of key skills tests seems to be a major factor 
in the formal non-completion of apprenticeships, even though 
the trainees in question become competent workers, attain their 
NVQs and remain in employment. As such, key skills tests would 
seem to require either immediate reform to ensure that they 
are operated consistently throughout the sector, or abolition. 

A further particularly disliked feature of both key skills and basic 
skills tests, by tutors and learners alike, is the generic nature of 
the tests. Key skills and basic skills teaching usually attempts to 
‘embed’ such skills in vocationally-relevant tasks, for example 
learning about ratios through calculating additives to the cooling 
system in a motor vehicle. However, the tests are generic because 
they have to apply to a wide range of candidates across different 
vocational sectors: for example, testing ratios by reference to 
cake recipes; not surprisingly, MVE apprentices do not always 
take kindly to being asked questions about cake recipes): 

The big drive [is] on ... embedding basic skills, making things 
contextualised - and then the qualifications aren’t! 

FE college programme manager 

Key skills and competences are also thought by many to be better 
assessed on-the-job rather than through being identified and 
assessed separately: 

Get rid of key skills altogether. Because there 's problem-solving . . . 
built in [ to the standards ] ... when he 's done some repairs he 
automatically covers problem-solving. Now if my student has 
completed unit 12, diagnostic and repair, those two tell me has 
sorted problems out for the customers. He ’s done it ... 
communication skills - that student cannot do that task unless 
he gets some data or information from somewhere or he talks 
to somebody. He can talk to his employer, he can talk to the 
customer, so there’s communication going on there ...he’s doing 
communication. 

FE college MVE head of department 

In this, the respondent seems to be absolutely at one with the 
original NVQ philosophy, but doubts over coverage and enforcing 
standards have clearly led policy-makers, and in turn awarding 
bodies, to develop hybrid approaches to assessing such skills. 



Section 4.8 



The performance-evidence-competence continuum: 
validity, reliability and equity issues 



We have hinted at many issues of validity, reliability and equity 
already. Tutors, supervisors and assessors at local level go to 
great lengths to support candidates, but often particular 
interpretations of standards or a lack of resources or workplace 
opportunities for assessment can compromise validity and 
reliability. 

Thus what one might call ‘opportunities to verify’ vary greatly 
across work-based and college-based settings. For example, small 
garages may not provide NVQ Level 3 opportunities to conduct 
diagnostic work with the latest computer technology. Equally, 
however, and somewhat ironically, well-resourced main dealers for 
leading car makers do not always provide NVQ Level 2 
opportunities for basic repair: 

How often does a Toyota go wrong? Some of these kids on these 
programmes, it’s really difficult to get the evidence. 

MVE lead verifier 

Volvo gearboxes don’t go wrong. Now the standards say he must 
show four pieces of evidence for transmission so we have to do 
a drives haft, clutch and gearbox, in two years he’s been there they 
haven ’t had a clutch or a gearbox in the whole place to do ... the 
standard says no simulation for that unit. He 's got to wait until one 
comes in. Now what if that’s 10 years? 

FE head of MVE department 

This can also lead to some boredom and disillusionment for 
trainees who do not experience sufficient variety of work. Because 
modern cars are so reliable, repair work can be very limited, and 
usually involves replacing parts or whole assemblies rather than 
actually working on them and repairing them: 

Another reason they drop out I think is I don 't think they get the 
variety of jobs ... a lot of the work that most of the garages are 
doing is very mundane. It’s the odd service, set of brake shoes, 
it isn't very often they strip the engine down. 

FE college head of MVE department 

In such circumstances, local colleges either provide ‘real’ repair 
work in the college by getting apprentices to work on staff cars or 
sometimes orchestrate temporary exchanges of apprentices 
across garages to extend their experience of different work. On 
occasions, however, this has led to more problems than it solves, 
as the ‘new’ apprentice may be more or less competent than the 
‘old’ one, leading employers to try to poach the better one. 




LSRC research report 



Section 4 



page 70/71 



In Sport & Recreation NVQ small hotel leisure facilities can be 
very limited in the equipment available, and indeed in client 
activity, so simulation is often called for, with another member of 
hotel staff acting as a client for the purposes of assessment. But 
doing a ‘fitness appraisal’ on a colleague who you know well, is 
clearly going to be very different from doing one on an unknown 
guest or visitor. Often, therefore, ‘ways and means’ are found to 
observe and verify competences, but this can stretch claims to 
validity to the limit. A related issue with respect to ‘opportunities 
to verify’ is that candidates may not necessarily be in a position to 
gather relevant evidence. For example, candidates already have 
to be in a supervisory position to demonstrate a lot of Level 3 
competences in Sport & Recreation, but they wouldn’t be in such 
a position and cannot secure such a position, if they are not yet 
considered competent: 

One of the issues we Ve had ... a learner who ’d completed the 
operational services at Level 2 and therefore wanted to progress 
onto Level 3 but as yet was not in a supervisory position. 

Sp&Rec NVQ assessor 

Similar issues pertain to client safety in Social Care, even at lower 
levels of the awards. Ideally care workers will not be in a position 
to exhibit evidence of safe practice until they are already 
competently safe. Similarly, emergency situations cannot be 
engineered, they can only be simulated: 

With the NVQ . . . direct observation is obviously essential . . . 
there will always be direct observation, unless it’s specified why 
you don 't have it ... for example, if it was about certain health 
and safety issues, you wouldn't wait until the place was burned 
down to show that you know how to evacuate the building. . . 

You have simulation. 
internal verifier, Social Care NVQ 

The policy issues here are whether current national standards are 
still appropriate to workplace activities (and if not, how they can 
be updated quickly) and in turn whether, and if so to what extent, 
simulation is acceptable. Changing a clutch in a college workshop 
because no such job has occurred in the workplace would seem to 
be acceptable simulation (if still deemed necessary), whereas 
‘pretending’ to do a client fitness appraisal on a colleague that 
one works with every day is clearly less appropriate: 

The main [thing] is ... using real people ... [for] ... simulations. 

I always say to them that they can bring somebody in if they want 
to ... in the past we ’ve done that and it seems to work better . . . 
it’s somebody who doesn't know them and is not used to them, 
somebody who doesn 't expect certain things to happen because 
students are expecting the same thing, they know what they 
should be doing. . . 
internal verifier Sp&Rec NVQ 

As discussed in Section 4.3, perhaps simply providing candidates 
with the option of a defined practical test would be the simplest 
solution to these problems. 



Other examples of problems with validity and reliability have been 
encountered. In particular, issues with respect to the intermediary 
power of external verifiers and moderators have been raised. 
These agents are crucially important fulcrums of the system and 
sometimes their interpretations can frustrate: 

[The moderator] came in and wanted to see an 11-a-side football 
game, when we only had five A-level students, and we had to pull in 
other students. He said, ‘They are not really A-level standard, ’ and 
we had to say ‘Well, they are not all A-level students. ' 

A-level tutor 

I was doing this presentation on evidence, and somebody stood up 
and said well we can't get transmission evidence, our external 
verifier won 't allow that. I said well what do you mean. .. So he said 
well we 're doing a clutch on an Escort, we 're not allowed to use 
that. Our E V has told us we must have a piece of evidence off a 
bigger car, that's more difficult to do than an Escort. I said well 
standards don 't say that. . . Oh but the EV thinks the Escort is too 
easy or the Fiesta is too easy, he has given us a list of vehicles 
that he would like... 

FE head of department and external assessor 

The notion that NVQs are national is just barmy... Every different 
place we walk into - they have a different notion of what it is. The 
classic is the amount of evidence required for each PC. So you go 
to some places and they’ll say there must be 3 pieces of evidence 
for each PC. Well who says that then, well nobody, but that’s our 
level . . . probably if you track it back it’ll be some EV saying it 
somewhere. 

training agency manager 

Providing too much learner support can also threaten validity; not 
so much in terms of ‘traditional’ concerns about reliability and the 
authenticity of the evidence produced, but in terms of the overall 
congruence between programme procedures and workplace 
demands. We have already noted the popularity of e-testing 
among learners, along with problems of the validity of testing 
‘literacy’ through use of point-and-click mouse technology. Some 
tutors wondered openly about the supposed equivalence with 
GCSE and the disservice potentially done to learners who might 
not be able to cope with extended writing if it were required of 
them in the future. Other tutors commented on the culture of 
drafting and re-taking assignments and unit tests in AVCE when 
such programmes were supposedly preparing learners for the 
transition to work: 

How vocational is that? Give me one work setting where there are 
no real deadlines and where it doesn’t matter if you take ages and 
ages to get it right. 

AVCE tutor 

These are issues of programme structure and assessment 
design, rather than the attitudes and practices of individual 
tutors. 
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13 Savory, Hodgson & Spours (2003) 
use the term ‘academic drift' to 
summarise similar findings from their 
Nuffield-funded study of Curriculum 
2000 and the introduction of AVCEs. 



Similarly, some tutors and assessors noted what we might term 
a ‘comprehension drift’ - ie writing about Business or Sport & 
Leisure, rather than doing Business or Sport & Leisure, and thus 
essentially testing language and comprehension rather than 
practical competence . 13 Similar views were expressed about the 
development of vocational GCSEs: 

True vocational education embraces employers, and is almost 
employer-led . . . but . . . Instead we have a sort of tenuous work 
experience. . . And then the government have fudged it totally, they 
have gone for these VGCSEs. That’s an anomaly in itself. All they 
are is academic subjects looking at a vocational area. A GCSE in 
Travel and Tourism is really a study of the travel and tourism 
sector, it’s not vocational. 
awarding body manager 



There are echoes here of our earlier observation about a lack of 
clear vision for vocational education. At the same time, however, 
writing about activities may also be a feature of a more general 
drift towards Quality Assurance performativity - being seen to 
prove that you’re doing something, rather than just doing it, or 
even irrespective of whether you are actually doing it or not. 



Section 4.9 



Lost in translation? Interpreting the language 
of assessment 



Another element of discussions of validity, reliability and equity is 
the enduring opaqueness of the outcomes-based, competence- 
based language of many LSS awards. While improvements have 
been made to many of the original specification documents, 
successive generations of learners, and indeed assessors, have 
to learn the language anew: 

The candidates will all tell you that it is understanding the 
language of the actual award itself... City and Guilds have made 
valiant efforts on various occasions to simplify the language and 
format of the standards, and it is easy for me ... but that is one 
of the big things that they will tell you. . . 
external verifier, Social Care NVQ 

We induct them over a day, and we try to keep it as simple and 
straightforward as we can, we repeat the information and try to ... 
just reassure them. I remember my first day on NVQ - it sounded 
like gibberish, and it is very wordy. I have found that people don 't 
really understand it until they have done one or two units. . . 
internal verifier, Social care NVQ 

That one I did there, it said ‘relevant available information on the 
communication differences likely to be experiences obtained from 
the appropriate sources and appropriate times’, so I simply ask 
the question, ‘What is communication, what are different forms of 
communication? Think of A and B and the way they communicate 
and then talk about it. 
internal verifier, Social Care NVQ 

Paradoxically, because NVQs also demand that routine workplace 
competences are demonstrated, including such things as ordinary 
health and safety precautions, some questions and behaviours to 
be demonstrated can be very patronising: 

The health and safety bit at the beginning I thought was 
pointless ... the fire exit is in the building I've worked in for 
a couple of months and obviously know, what's the point in me 
sending that off to get assessed ... [ and] I think they should 
look into your qualifications before they test you because it was 
quite insulting being asked what a monitor was. . . 

Sp&Rec NVQ candidate 

The only way round this is to say, 7 know this is silly but do you 
mind explaining...’ This gives a very bad impression of the NVQs. 
internal verifier, Social Care NVQ 

Such problems derive not only from the language of competence, 
criteria, range statements and the like, but also from the generic 
terminology designed to encompass different specific areas of 
work. In this extract from an observation, a Social Care candidate 
is confused by the generic term ‘client’: 

Assessor Why is it important to review the placement with the client and why 
is it important to provide the clients with positive feedback on their 
achievements? 

Candidate When you say client, is that the child? 
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We have already seen how college tutors support learners by 
providing detailed mark schemes, assignment templates and so 
forth, while workplace assessors support learners through the 
process of identifying evidence. This can lead to the sort of narrow 
instrumentalism discussed above, with students learning to 
accumulate marks rather than accomplish worthwhile tasks: 

There are issues around the interpretation of assessment 
criteria. . . Students find the whole . . . business very difficult to 
understand. In the end, they just come to understand how to get 
marks . . . and end up in a position of simply needing to know, 

‘What do I have to do?’ 

AVCE tutor 

There are many other examples in the data of tutors and 
assessors interpreting awarding body specifications and criteria 
and providing simple translations of what they ‘really mean’: 

I’ll say the questions as they are, if [they] don't understand them, 

I ’ll try and re-phrase them in English. . . 
assessor, Social Care NVQ 

I had this book and they did exactly the same that I had to do ... 
they had to put the PCs down and then put underneath it, in 
everyday language what it really means . . . and I thought, wow, 
it’s not just us ... 
internal verifier, Social Care NVQ 

Equally, however, assessors themselves can sometimes be 
confused: 

The language, it’s a big problem, not just with the staff who are 
doing the NVQ but sometimes with the assessors as well, if we 
didn 't have our moderation meetings once a month, then try 
to decide what that actually means they’d have lots of problems. 
internal verifier, Social Care NVQ 

Clearly, there are major implications here with respect to 
the interpretation and mediation of assessment procedures 
and practices at local level. There are also fundamental 
epistemological issues with respect to the nature of the 
‘knowing’ that such assessment produces: in what sense do unit 
specifications, range statements and so forth ‘represent’ the 
reality of workplace competences if they are not recognised as 
such in the workplace, have to be ‘translated’ into ‘plain English’ 
for workplace use, and then observed competences have to 
be translated back again into acceptable evidence statements? 



Section 4.10 



Local ‘communities of practice’ - interpretive, 
mediating, local culture(s) of education, training, 
employment and assessment 



Effective criterion-referencing and its workplace corollary, 
competence-based assessment, depend on clear and detailed 
statements of programme specifications and the criteria 
by which they are deemed to have been met. Validity and the 
appropriateness or fitness-for-purpose of assessment methods 
are considered of more importance than reliability; reliability is 
assumed to ‘take care of itself if assessments are valid and 
judgements made in accordance with extant criteria. Thus 
national standards are ensconced in the specifications and 
criteria, and enacted and judged at local level. 

As we have already seen, however, considerable variation exists 
at local level in terms of both the general assumptions that 
tutors and learners make about the nature and purpose of 
assessment, the specific resources available, and the procedures 
and practices that develop. Thus there is significant variation 
in the amount and quality of support provided for learners 
through the assessment process - the same regulations can 
be interpreted in very different ways as awarding body guidelines, 
procedures and performance criteria are distilled by tutors and 
assessors into understandable, accomplishable activities, and 
learners given support and feedback in these terms. Sometimes 
this is manifest in detailed mark schemes and assignment 
templates. Sometimes it is manifest in intuitive judgements 
and practices - helping a ‘good lad’ get through ‘the paperwork’. 
Some vocational sectors and assessors/verifiers seem to regard 
this as a significant problem: 

Managers or the supervisors at the facility may well be 
conversant with the standard operating procedures of their 
individual facilities, they are not fully conversant with the 
national standards ... for the NVQ . . . certainly witness testimonies 
don’t tend to prove a valid piece of evidence. 

Sp&Rec external verifier 

Others, however, see the issue more as an inevitability, and, in a 
supposedly employment-led activity, a problem that ought simply 
to be addressed by more employer involvement in work-based 
assessment: 

At the end of the day the employer is the best person to say 
whether that person is competent or not. He 's got him for 6 days 
of the week... So if that person is doing a good job it’s the 
employer who really needs to sign him off in my opinion. 

FE college head of MVE department 
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The divergence of perception and opinion expressed here may 
derive, at least in part, from the history of the two vocational 
fields. Sport and Recreation is relatively new, and still emergent 
as a major industry and field of training. Perhaps trainers and 
assessors feel that they have to ‘attend to the traditional’ in 
assessment design. Certainly we have seen how over-assessed is 
the ‘Progression Award’ in this field. Motor Vehicle Engineering, on 
the other hand, has developed over 100 years, and perhaps 
therefore includes more practitioners who are confident of their 
own and others’ judgments. Both Stasz et al. (2004) and Fuller & 
Unwin (2003) note similar tendencies in what we might term 
‘emergent’ and ‘traditional’ fields, with Stasz et al. (p31) reporting 
higher completion rates for apprenticeships in fields where they 
have traditionally been offered and accepted as the appropriate 
way of entering skilled employment. Additionally, no-one in a local 
car-using community of college tutors, employers, learners and 
their friends and families has an interest in unsafe apprentices 
being certified to practice; so it may be that this particular 
‘community of practice’ will be more vigilant, and will need less 
centralised oversight and control than some others. 

This issue arises in two forms/contexts: 

■ What we termed in the introduction the ‘vertical community of 
practice'- delineated, as the case may be, by industrial practices 
and/or academic disciplines; these define and embody the 
assumptions which are made about what counts (or should count) 
in gaining an award, communicated from awarding body, 

to assessors/external verifiers/moderators, to curriculum 
managers/supervisors, to tutors/trainers, to learners. 

■ What we might term the local, geographical, horizontal community 
of practice involving college-based tutors, assessors, training 
agencies and employers - the context of operation and culture 

of support for different ‘sorts’ of learners. 

How the two interact was briefly discussed in the opening part 
of the report in terms of ‘reification’ and ‘participation’. In this 
respect, the research has identified the self-selection and 
induction of learners into different ‘communities of practice’. 



The policy issue is the extent to which these communities of 
practice are part of the problem or part of the solution with 
respect to defining and ensuring ‘national’ standards. The vertical 
community is obviously central to policy; the horizontal local 
community less so, with variation at this level having been seen 
as something which must be written out of the system (literally) by 
ever more detailed specifications. Yet it is the horizontal local 
community which realises standards in action. Perhaps central 
control has run its course, and rather than trying to extinguish 
local judgement it is now time to recognise it as inevitable, but 
seek to improve it by supporting it. Fuller & Unwin (2003, p422) 
similarly conclude that the development of a much more self- 
consciously integrated pedagogy of apprenticeship in college and 
workplace - rather than further development and reificiation of 
standards - is the way to improve the quality of both the process 
and outcomes of apprenticeship. Likewise Stasz et al. (2004, 
p64) identify local capacity-building as the most important factor 
in improving processes and outcomes. Certainly there are 
very significant Continuing Professional Development (CPD) 
opportunities to be exploited by awarding bodies, not necessarily 
as providers of CPD, but as commissioners and orchestrators, 
legitimating a new approach. Their involvement would signal that 
they were happy to view CPD as involving more than training in 
compliance with AB procedures; rather it should involve the 
strengthening of local judgement, and furthermore their regional 
structures could facilitate local groups drawing on the experience 
of parallel communities elsewhere. 
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Innovation without change? The enduring 
academic/vocational divide and the need for a national 
vision for vocational education 



A significant factor in the development and implementation 
of policy and practice with respect to the assessment and 
certification of learners in the LSS is the constant level of 
innovation and perturbation in the system. This is the case 
with respect to all aspects of the system involving: 

■ the institutional and financial arrangements under which colleges 
and other training providers operate, including new forms of public 
sector management and the use of targets to drive change 

■ awarding body organisation (including mergers and increasing 
commercialisation) and the national quality assurance regime 
under which they now operate 

■ the details of programme specifications and assessment 
methods employed in the sector. 

Additionally, and more specifically, the provision of general 
vocational education in schools and colleges has been subject to 
continual re-organisation and the production of an ‘alphabet soup’ 
of successive initiatives over the last twenty years: CEE, CPVE, 
GNVQ, AVCE. 14 

Many of these developments relate to, and in some key respects 
derive from, the issues identified in the introduction. In particular, 
a full political commitment to any sort of properly comprehensive 
education system only came about in the mid-1980s with the 
absorption of O-levels and CSE into GCSE, and even then only up 
to 16 years of age. This was closely followed by advocacy of 
competence-based training and assessment. As such it is 
perhaps not surprising that many aspects of current policy and 
practice are still very much a ‘work in progress’ and that 
operations at local level can be hard-pressed. It is important to 
recognise this and posit findings in terms of the options and 
trade-offs that any assessment regime must deal with. 

At the same time, however, all this innovation does not seem to 
have had much of an impact on the basic problem of a lack of 
status for vocational education and training, and the lack of a 
national vision for what we want from vocational education and 
training. Change applies far more to the vocational track than to 
the post-compulsory academic track. As one of the college tutors 
put it, there is ‘endless, endless change, constant change,’ in the 
further education sector generally and in vocational education 
especially. The contrast of constant change in qualifications and 
assessment in vocational education with the stability of teaching 
and assessment in GCE A-level, despite the changes to AS and 
A2 in Curriculum 2000, is an important contextual factor in 
this study. 

It is also worth noting that this change brought pressure at local 
level which affected, and in some key respects restricted, 
fieldwork access, with colleges being unable to cooperate as fully 
as might have been desirable, and some independent training 
agencies being uninterested in cooperating at all. Inspection also 
had a negative impact on fieldwork access, as colleges and 
individual tutors indicated that they really could not cope with any 
further investigations or, as they saw them, intrusions. Research 
in the LSS seems likely to become more difficult to negotiate and 
accomplish in these circumstances, just as the necessity for it 
becomes more apparent. 



14 Certificate of Extended Education, 
Certificate of Pre-Vocational 
Education, General National 
Vocational Qualifications, 
Advanced Vocational Certificate 
of Education. 




Enduring social reproduction, in the context of individual 
success and achievement 

There is also a need, finally, to recognise the importance of 
different levels of analysis in the report, and the implications this 
has for discussing findings. There is overwhelming evidence of the 
enduring nature of the academic/vocational divide, with relatively 
low levels of status/expectation in the vocational sector, and the 
self-selection of learners into communities of practice which 
match the ‘assessment career trajectories’ with which they feel 
most comfortable. This might be considered to be disappointing, 
even depressing. Yet within this broad picture of continuing social 
reproduction, there is also evidence of widespread individual 
success and the opening up of opportunities for the future. Tutors 
and assessors alike work hard to support learners through 
assessment processes and the accumulation of marks and 
evidence made possible by the move towards criterion-referencing 
and outcomes-based assessment has certainly benefited 
achievement and progression. More, still, could be accomplished 
at this level with widespread dissemination of coaching guidance, 
exemplary tasks and activities, and so forth. And insofar as some 
element of external testing is considered important to 
qualification in particular vocational and skill sectors, electronic 
testing is much preferred by learners when compared with paper- 
and-pencil tests. Such developments cannot address the more 
structural issues of the status of vocational educational and the 
vision that we wish to develop for the future, but they can certainly 
assist individuals to succeed in current circumstances. 
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Conclusions and recommendations 



We have compared and contrasted the assessment experiences 
of learners in different settings and attempted to identify what 
assessment regime works best in enabling learners to progress in 
which contexts and in which sectors. The comparing and 
contrasting have been particularly instructive. It is apparent that 
there are very important differences in the organisation of 
assessment activities and the opportunities open to learners 
across different sub-sectors and assessment settings. Major 
anomalies and contradictions in the structure of awards, the 
assessment methods employed, and the approach of assessors 
and moderators have been identified. Significant paradoxes 
have been revealed, such as: 

■ the pressure to maintain standards of academic excellence at 
A-level while also increasing the numbers who pass and achieve 
good grades 

■ the use of formative feedback throughout the sector which could 
nevertheless prevent the award of ‘distinctions’ in AVCE since 
these can only be gained by independent work 

■ the problem of reconciling authentic vocational assessment with 
limited provision of workplace tasks and the use of simulation 

■ the move towards embedding basic skills in vocational 
programmes but assessing them through generic test items 

■ the use of multiple-choice tests (both paper and electronic) to 
assess ‘literacy’ at higher basic skills levels, without any further 
requirement for extended writing 

■ and, perhaps most unfair and revealing of all, the denigration of 
good GCSE grades achieved by some groups (apprentices, who 
must ‘replace’ them with relevant key skills achievements after 
three years) compared to others (eg A-level takers who can still 
use them for university entrance and/or job applications many 
years later). 

Yet it is also clear that the move towards criterion-referenced 
assessment, and its vocational sibling, competence-based 
assessment, which has underpinned the move towards greater 
transparency of intended learning outcomes and the criteria by 
which they are judged, has significantly benefited learners in the 
LSS in terms of the numbers of learners retained in the system 
and the awards which they achieve. 

Clarity in assessment procedures, processes and criteria has 
underpinned the widespread use of coaching, practice and 
provision of formative feedback to boost individual and 
institutional achievement. Coaching, practice and the possibility 
of retaking module tests to improve grades, have boosted A-level 
awards. Coaching, practice and the constant drafting and 
redrafting of assignments, have underpinned the development of 
AVCE and comparable Level 3 programmes such as the Access 
courses which we investigated. Similar approaches, but broken 
down into even smaller steps, have developed in Adult basic skills 
teaching and assessment. Use of assessment events in NVQ, 
notjustforthe assessor to observe trainee performance, but to 
interact with the trainee in order to feed back advice and elicit 
and organise emergent evidence for portfolio completion, is 
widespread across the vocational training sector. 




These are practices which have developed out of the clarifying of 
objectives and the intimate relationship between assessment, 
pedagogy and learning. They greatly help learners to achieve. In 
this respect, a key finding of the research, as reported in Section 
4.3, is that it is not so much assessment methods per se which 
make a difference to learner achievement, but rather their 
congruence with learner perceptions of appropriateness to the 
learning task at hand, and the opportunities they provide for 
detailed communication of required knowledge and competences. 
Transparency of procedure and criteria are most important, 
irrespective of the method employed. This applies as much to 
the language of competence and the elicitation of evidence 
(Section 4.9) as to multiple-choice tests. 

In saying this, however, we have perhaps identified the greatest 
paradox of all, the symbiotic relationship between transparency 
and instrumentalism. The clearer the task of how to achieve a 
grade or award becomes, and the more detailed the assistance 
given by tutors, supervisors and assessors, the more likely are 
candidates to succeed; but succeed at what? Are we now content 
to accept assessment as learning ? Wherein does the challenge 
of learning reside? From where does an intrinsic sense of 
achievement arise? Where is the overall, holistic vision of what it 
is to understand ‘business’ or become a competent and confident 
motor vehicle technician? The research team members 
themselves are rather divided on the matter. Some see what we 
might term as the ‘cognitive costs’ of incremental and 
aggregatory achievement being outweighed by the social and 
economic benefits which accrue to individuals in terms of 
improved achievement, employment prospects and life chances. 
Others are less sure, and wonder about whether this isn’t, 
actually, dumbing down, after all: just because the Daily Mail 
complains about it, doesn’t mean it isn’t true. 

The issue is not just to do with criterion-referenced and 
competence-based assessment, however. It also derives from 
the current accountability context in which assessment operates. 
Helping learners to achieve in the ways we have identified could 
be carried out in a much more educational context in which the 
goal really was learning and self-development, rather than the 
inexorable certification of individuals and the measurement of 
systemic efficiency. Perhaps the more important general finding 
here is that no approach to or method of assessment is immune 
from distortion when too many consequences ride on the results. 
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With respect to more specific findings and recommendations, 
it is apparent that even in current circumstances, positives are 
identifiable, while problems also remain to be addressed. Thus: 

1 Assessment methods perse do not directly affect learners’ choice 
of award or likelihood of success, but the association of certain 
awards with methods which employ extensive writing (coursework 
assignments, exam essays) does. Thus for example, practical 
tests and/or multiple-choice tests are seen as acceptable - and 
indeed unavoidable - across most groups of learners in the 
sector, especially younger trainees, but extensive written work is 
disliked and largely avoided, except by A-level takers. Even in 
school and college-based AVCEs, the view is emerging that these 
are becoming too based on writing about the vocational field being 
studied, rather than engaging in the practical development 

of competence. 

2 Detailed tutor and assessor support, in the form of exam 
coaching and practice, drafting and redrafting of assignments, 
asking ‘leading questions’ during workplace observations, and 
identifying appropriate evidence to record in portfolios, is 
widespread throughout the sector and is effective in facilitating 
achievement and progression. In current circumstances the 
further development of such strategies, underpinned perhaps, 
by more formal recognition of them by ABs as acceptable, and 
provision of relevant and appropriate AB guidelines and CPD 
material in order to address equity issues, is likely to lead to 
further individual success and rising numbers of awards 
achieved - see also 10, below. 

3 Further development of such strategies would also be helped by 
rendering one-year or two-year programmes more accomplishable 
through credit accumulation (eg the Progression Awards and 
NVQs which we studied). Achieving formal, certifiable ‘stepping 
stones’ along the way of an NVQ Level 2 or Level 3 would 
probably increase retention and completion rates. Such credit 
accumulation would also render NVQs more directly comparable 
with other awards at the same level and possibly facilitate credit 
transfer across awards. Discussions over credit accumulation are 
under way between ABs and QCA, and a policy decision should be 
taken as quickly as possible so that effective development work 
can begin. 



4 A longer-term implication of 1-3 above is that inconsistencies 
across the sector should be reviewed, and a wider range of 
assessment methods should be employed across all awards, with 
QCA and ABs allowing more candidate choice with respect to 
method of assessment. This could further mobilise learner 
engagement with the assessment process, while also addressing 
the anomaly of, for example, A-level takers being able to replace 

a module test with an assignment, or vice-versa, while NVQ takers 
must produce evidence-in-action, even if only through a (poor) 
simulation, when taking a practical test might be both more 
reliable and more welcomed by the candidates. Currently ‘fitness- 
for-purpose’ arguments justify AB provision of methods, yet, as we 
have seen, actual provision emerges out of a much more murky 
set of compromises between assessment theory, policy and 
practice. In a social and economic environment which supposedly 
privileges the consumer over the producer, perhaps consumer 
choice in LSS assessment, coupled with the need for equality of 
consumer choice across the sector, should be given more weight 
when such compromises are being struck. A range of assessment 
methods could be made available for all awards, with the 
candidate choosing that combination of methods which most 
suits their learning style and maximises their chances of success. 

5 The same issue of consistency and candidate choice arises with 
respect to the shelf-life of GCSEs and the necessity for some 
candidates in the sector, but not others, to take key skills tests. 
Why are key skills mandatory in the further education and training 
sector but not within the academic A-level track? Why are 
GCSEs no longer considered to be valid and reliable indicators 

of achievement after three years for apprentices, but are 
so accepted for everyone else? Key skills tests are not serving 
the purposes for which they were ostensibly designed, and should 
be reformed to become uniformly applicable across the LSS, 
or abolished. 

6 In contrast, e-testing has proved popular and effective where 
we observed it. Wider use of e-testing across programmes and 
via wireless technology in workplaces could improve completion 
rates, pass rates and speed of progression in ‘underpinning 
knowledge’ tests and adult basic skills. Piloting and evaluation 
would be an appropriate way to develop, being sensitive to issues 
of learner IT capability. Equally, however, such development would 
have to be sensitive to issues of validity. This further underpins 
the need for a wider range of assessment methods to be 
employed across all awards in the LSS. 
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7 The ‘wider benefits of learning’ are very apparent in the data but 
are not currently identified or recorded systematically. Pursuing 
and recording them more specifically could go some way to 
counteracting the narrow instrumentalism noted above, and the 
pressures of accountability which ultimately drive such 
instrumentalism. It might also, if it were developed across the 
sector, feed into the wider debate about what we wish to achieve 
through vocational education. Recording work-based learning 
across the sector, rather than key skills test results, for example, 
might help to refocus attention on the purpose of post-compulsory 
education as well as measure its outcomes with more validity. 

The recording of wider competencies could be pursued fairly 
easily, since evidence is likely to exist already - eg evidence of 
writing CVs and job applications, attending interviews, helping 
with children’s homework and after-school clubs, etc. The 
recording of increasing confidence and/or social capital would be 
more challenging but might be an interesting topic for a pilot study 
by an AB - identifying and accrediting the collective social 
achievements of a community involved in a Sure Start programme 
for example, rather than just the particular achievements of 

15 South Yorkshire and Humber OCN individual Candidates. 15 
have explored similar possibilities 

8 Whether or not the more radical implications of 4, above, are 
accepted, with respect to consumer choice of assessment 
methods generally, attention must be paid to key issues of 
comparability and equity, such as variations in workplace 
resources and ‘opportunities to verify’. A poor simulation is no 
substitute for a well-designed practical test which could be carried 
out in better-resourced locations (including colleges of FE) and 
which candidates would probably prefer in any case. The tyranny 
of what we might term ‘NVQ competence theory’ which insists on 
assessment in situ must be replaced by a more pragmatic 
approach to issues of validity and reliability, with FE colleges and 
other major local infrastructures (such as local authority sports 
facilities, in the case of Sport and Recreation) being employed as 
necessary and available for practical tests. Electronic simulation 
via virtual reality technologies could also be employed where 
appropriate, eg for analytic tasks such as client fitness 
appraisals, though probably not for more obviously hands-on 
tasks such as replacing a faulty clutch. 



with a small pilot on Group 
Accreditation (see Hawkins, 
no date) and Sanders (1995). 



9 Similarly, the numbers of visits which assessors and verifiers 
make to candidates, and the importance of the quality of 
the relationship when such visits take place, must be addressed. 

In principle NVQ assessment is intended to take place when 
candidates are ‘ready’. A workplace supervisor should be in the 
best position to judge this and either conduct the assessment 
themselves or call in the relevant person to do it. In practice the 
working schedules of supervisors and especially of other 
assessors who move around a series of workplaces do not easily 
allow this. Candidates are assessed and their portfolios amended 
according to the visiting schedule of the assessor, whether 
or not candidates are ready, or perhaps, have been ready for 
several months. Training agency assessors working with MVE 
apprentices seem to be particularly elusive in the cases we 
studied, though similar problems arose with at least one social 
care training agency. By contrast, assessors who were also 
local college lecturers and saw apprentices both at college and 
in the workplace were probably best placed to conduct regular 
assessments and develop high quality relationships with 
trainees. QCA, the Adult Learning Inspectorate (ALI), Sector Skills 
Councils (SSC) and ABs should highlight the issue and provide 
examples of good practice for centres. ALI’s ‘Excalibur learning 
network’ may be helpful here. It already provides examples of good 
practice and using this would provide a sector/regulator focus. 
This is especially urgent with respect to the introduction of e- 
portfolios and the possibility that assessors and verifiers may 
come to operate almost wholly at a distance, especially those in 
commercial training agencies who may be faced with even more 
pressing financial targets than college-based assessors. 

10 The problem of the frequency and quality of the assessment event 
brings us back to the issue of national, comparable standards and 
the role of the local, horizontal, ‘community of practice’, be it 
composed of subject-specific tutors and moderators within and 
across local colleges, or tutors, employers and assessors within 
a regional, vocational sector. Central prescription of national 
standards in academic subjects and vocational fields has been 
prominent for more than 20 years. Some would argue that such 
prescription is necessary, and has in any case arisen from 
widespread involvement of practitioners and employers in 
analysing national needs and specifying appropriate outcomes 
and competences. Thus such definitions of national standards 
represent widely agreed ‘good practice’. All recent studies, 
however, be they research reviews (Stasz et al. 2004) or new 
empirical investigations (Fuller & Unwin 2003) point to the fact 
that central prescription of ‘standards’ has run its course. The 
present study confirms this. Centrally-orchestrated analysis of 
sector needs and associated definitions of what local 
practitioners should be trying to achieve may still be pursued 
and produced. But lack of sufficient, relevant and appropriate 
resources at local level, coupled with the inevitable need to induct 
successive generations of trainers and assessors and the 
unavoidable interpretation and mediation of national criteria 
which this entails, means that further improvement of both 
the numbers of successful candidates, and the quality of the 
experience and awards they receive, will be dependent on 
capacity-building at local level. 
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The provision of clear national programmes of study, the intended 
outcomes of training, and the criteria by which success can be 
judged, have certainly led to increasing numbers of learners 
achieving awards by the means we have identified. But this very 
process has led to too narrow a reliance on accumulating marks, 
or elements of competence, and a narrowing of the quality of the 
learning experience. Moreover, even the closest attendance to 
objectives and criteria cannot expunge local variation; on the 
contrary, it conceals it, and also conceals the potential unfairness 
of some of the judgements made. Further specification of 
assessment detail is only likely to exacerbate the problem. What 
is required is an acknowledgement that local communities of 
practice are the context in which all meaningful judgements are 
made, and thus should be the level of the system at which most 
efforts at capacity-building are directed. Such local communities 
of practice are also the contexts of action in which holistic visions 
of achievement and career development in particular fields are 
enacted and made manifest to learners. Recognising assessor 
interaction with candidates as inevitable, and encouraging it as 
part of the learning experience ratherthan attempting to ‘write it 
out’ of the process, would be an important first step. Thereafter, 
its further development and capacity to accomplish fair 
assessment for learners will depend on the provision of local 
and regional CPD which draws on the experience of parallel 
communities elsewhere. Awarding bodies could be important 
brokers and commissioners of such CPD in tandem with local 
Learning and Skills Councils. Although it may be the case that AB 
involvement would reinforce ratherthan disrupt the expectation 
that training should simply focus on compliance, their involvement 
could also signal that they were happy to view CPD as involving 
the strengthening of local judgement, thereby legitimating a new 
approach, and their regional structures could facilitate local 
groups drawing on the experience of parallel communities 
elsewhere. Disseminating evidence from this report could even 
provide the focus for early discussions at regional level. 
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Appendix 1 


Methodology and summary of case study data sources 



Table 5 C&G NVQ Sport & Recreation levels 2&3 + Progression Award 

Methodology and ie workplace and college-based training & assessment 

summary of case study Product manager, curriculum managers, tutors, assessors, EVs 7 

data sources 

Learners/candidates 30 



+ document analysis & observation 
Resulting in 33pp case report 




C&G, NVQ, Motor Vehicle Engineering levels 2&3 focusing on FMAs & AM As 
ie workplace and college-based training & assessment 




Product manager, CV, LV, EVs, tutors, assessors, employers 


18 


Learners/candidates 


23 


+ document analysis & observation 
Resulting in 48pp case report 




C&G, NVQ, Social Care levels 2&3 
ie workplace assessment and accreditation 




Product manager, EVs, IVs, assessors, supervisors 


24 


Learners/candidates 


36 


+ document analysis & observation 
Resulting in 34pp case report 




AS & A2 PE& Sport, AVCE/BTEC Leisure & Recreation and Sport & Fitness 
ie sixth form and college-based AVCE and A-levels in vocationally-related subjects 




AB manager, HoDs, course leaders, tutors 


10 


Learners/candidates 


42 


+ document analysis & observation 
Resulting in 28pp case report 




AS, A2 and AVCE Business Studies 

ie college-based AVCE and A-level in vocationally-relevant academic subject 




Curriculum managers, tutors 


5 


Learners/candidates 


31 


+ document analysis & observation 
Resulting in 21pp case report 




NOCN accredited Access to Higher Education 
ie adults returning to learning in an academic context 




NOCN development officer, curriculum managers, tutors 


5 


Learners/candidates (including 6 access course graduates now on u/g courses) 


37 



+ document analysis & observation 
Resulting in 29pp case report 




C&G adult basic skills testing (NQF Levels 1 & 2; including online testing) 
& NOCN ABS testing (Entry levels 1, 2 & 3, and Levels 1 & 2) 
ie adults returning to learning in basic skills, largely college-based 
but including some workplace testing via wireless laptops 




Curriculum managers and tutors 


25 


Learners/candidates 


9 



+ document analysis & observation 
Resulting in 81pp case report 




NOCN accreditation of informal learning in adult and community education 
ie adults involved in community development activities via Sure Start and 
having informal learning accredited 



Programme organiser 


7 


Learners/candidates 


30 


+ document analysis & observation 
Resulting in 16pp case report 




Totals 




Total managers/supervisors/tutors/assessors/etc 


95 


Total learners/candidates 


237 


Total pp case report drafts 


320 



Additionally, interviewing has been taking place in the 
context of the Ufi-funded extension specifically to investigate 
e-assessment in Learndirect contexts. The summary of findings 
regarding e-assessment draws on this data in addition to 
examples observed in MVE and basic skills testing noted above. 

Similarly, the questionnaires (numerical data reported in 
Appendix 3) included open response comments on the best/worst 
features of the assessment methods and procedures which 
candidates encountered, and these are used in the final report 
as appropriate. 

The basic research design involved a series of parallel case 
studies of ‘assessment-in-action’ in the learning and skills sector 
(LSS), with the boundaries of each case being established with 
respect to particular qualifications/awards and the contextual 
and regional factors which influence the assessment of awards in 
practice, including awarding body procedures and processes. The 
aspiration was to collect data across a wide range of LSS contexts 
and awards in order to describe and analyse assessment 
practices across the sector. Thus the study is broad in scope but 
focused in terms of topic and depth of analysis. The intention is to 
learn lessons about assessment across sub-sectors and 
contexts, by comparing and contrasting experience of different 
awards and methods of assessment in different settings. 

MVE, Social Care and Sport & Recreation were selected to afford 
a cross-section of learners in terms of age, gender, race, and prior 
educational experience and attainment, along with a range of 
assessment methods and contexts of practice from relatively 
formal college-based settings, including formal knowledge testing, 
to less formal but still structured work-based settings (eg MVE 
apprentices), to informal work-based social care settings. 

A-level and AVCE PE and Business Studies were selected to afford 
the opportunity directly to compare assessment procedures, 
methods and experience across 16-19 academic and vocational 
subject areas, and establish some overlap across fieldwork 
activities for purposes of triangulation (eg Sport and Recreation). 
Gender, race, and prior educational experience and attainment 
also informed selection of learner respondents. 
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Access courses and various adult basic education settings 
were included to investigate learners’ responses to paper-based 
testing, online testing, other less formal assessment methods 
at Entry level, and contrasts between ‘separate’ teaching and 
assessment of basic skills and ‘contextually-embedded’ learning 
and assessment; along with assessment processes and 
progression opportunities into HE. As with fieldwork in other 
sub-sectors, age, gender, race and prior educational experience 
also informed the selection of learner respondents. 



Theoretical model and research design of assessment in 
the learning and skills sector 



Appendix 2 



Figure 1 
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Appendix 3 


Examples of questionnaires 



Questionnaires were distributed to a wider sample of learners in 
the case study sites derived from the same cohort of learners; 
ie, if 10 or 15 learners in a particular setting were interviewed, and 
their peer group consisted of a total of 50 learners in the same 
cohort or year group, the questionnaire was given to all of these 
peers in the same cohort. The intention was to validate and 
extend the interview data with the larger group. 




a 

Example for NVQ and related workplace learners 

This questionnaire aims to explore your experience of the 
assessment procedures for your qualification. Following analysis, 
results will be passed on anonymously to the awarding bodies. 

To encourage you to complete and return the questionnaire, you 
can be entered for our prize draw. Your answers will be anonymous 
but if you would like to be entered into our free prize draw to win 
£50, please include your name and contact details at the end. 
Your responses will not be connected to these details in any way. 

Please write your answer in the space provided or tick the 
appropriate boxes 

About you 

1 Are you? 

□ male 

□ female 

2 How old are you? 

3 How would you describe your ethnicity? 

□ White British 

□ Other British (please specify) 

□ Other (please specify) 

4 What is your first language? 

5 Have you any previous qualifications? 

□ None 

□ GCSEs - none at level C or above 

□ NVQ 1 or Progression Award 1 

□ GCSEs - less than 5 at level C or above 

□ GCSEs - 5 or more at level C or above 

□ GNVQs 

□ A-levels 

Other (please specify) 



About the qualification 

6 Please state which subject(s) and qualification(s) you are taking 







When did you enrol? 
Month 


Year 


When do you expect to finish? 
Month 


Year 
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9 Why did you choose this qualification instead of other options? 



10 Did anyone help you decide to enrol for this qualification? 

□ Yes 

□ No 

If so who? 

11 Do you feel you are making good progress? 

□ Yes 

□ No 

How do you know? 

12 Are you intending to take further qualifications after this one? 

□ Yes 

□ No 

□ Don’t know 

If yes, please state which one(s) 



13 Would you recommend this qualification to others? 

□ Yes 

□ No 

Please explain your answer 



About the assessment process 
14 How are you assessed? Please tick all that apply 

□ Written assignment 

□ Key Skills test 

□ Observation by assessor 

□ Witness testimony 

□ Spoken questions/interview 

□ Internal exam 

□ External exam 

□ Online tests 

□ Presentations 

□ Group work 

□ Simulation/mock situation 

□ Practical test 

□ Project work 

□ Video/audio recording 

□ Other 
Please state 



15 Did anyone explain to you the way(s) in which you were going 
to be assessed? 

□ Yes 

□ No 

If yes, who explained them to you? Please tick all that apply 

□ Assessor 

□ Supervisor 

□ Tutor/teacher 

□ Other (please specify) 

16 How often do you see your assessor for a formal assessment? 



17 How long does a formal assessment meeting with your assessor 
usually take? 



18 Do you receive help or guidance from your assessor 
at other times? 

□ Yes 

□ No 

If yes, please state where and when 



19 If your assessor is not a work colleague how do you contact them? 



20 Does anyone help you with difficulties apart from your assessor? 

□ Yes 

□ No 

If yes, please say who 



21 Which of the following best describes your view of the amount 
of support available to you in the run-up to being assessed? 

□ There is lots of support 

□ There is adequate support 

□ There is not enough support 

22 Which of the following best describes the quality of the support 
available to you in the run-up to being assessed? 

□ Support is of high quality 

□ Support is of satisfactory quality 

□ Support is of poor quality 
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23 We are also interested in which assessment methods you prefer. 
Please rank the top 3 in order of preference with 1 = first 
preference, 2 = second preference, 3 = third preference 

Written assignment 

Key skills test 

Observation by assessor 

Witness testimony 

Spoken questions/interview 

Internal exam 

External exam 

Online tests 

Presentations 

Group work 

Simulation/mock situation 

Practical test 

Project work 

Video/audio recording 

Other 

Please state 



About your ideas for improving the way you are assessed 

24 What has been most positive about the way you have been 
assessed for this qualification? 

1 

2 

3 

25 What has been most negative about the way you have been 
assessed for this qualification? 

1 

2 

3 



Thank you for your time. The sponsors for this research 
are City and Guilds and the Learning and Skills Research 
Centre and they will take your views into consideration 
when reviewing their awards. Please include your details 
if you want to be entered for the prize draw and return 
in the prepaid envelope provided. 

Name 



Contact details 



b 

Example for A-level and related college-based learners 

This questionnaire aims to explore your experience of the 
assessment procedures for your qualification. Following analysis, 
results will be passed on anonymously to the awarding bodies. 

To encourage you to complete and return the questionnaire, you 
can be entered for our prize draw. Your answers will be anonymous 
but if you would like to be entered into our free prize draw to win 
£50, please include your name and contact details at the end. 
Your responses will not be connected to these details in any way. 

Please write your answer in the space provided or tick the 
appropriate boxes 

About you 

1 Are you? 

□ male 

□ female 

2 How old are you? 

3 How would you describe your ethnicity? 

White British 

Other British (please specify) 

Other (please specify) 

4 What is your first language? 

5 Have you any previous qualifications? 

□ GCSEs - 5 or more at level C or above 

□ GCSEs - less than 5 at level C or above 

□ Other (please specify) 



6 Are you working in some kind of employment currently? 

□ Yes 

□ No 



If yes, how many hours per week on average? 


hours 


About the qualification 

7 Please state which subject(s) and qualification(s) you are taking 




8 When did you enrol? 
Month 


Year 




9 When do you expect to finish? 
Month 


Year 
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10 Why did you choose this qualification instead of other options? 



11 Did anyone help you decide to enrol for this qualification? 

□ Yes 

□ No 

If so who? 

12 Do you feel you are making good progress? 

□ Yes 

□ No 

How do you know? 

13 What grade(s) do you expect? 

14 Are you intending to take further qualifications after this one? 

□ Yes 

□ No 

□ Don’t know 

If yes, please state which one(s) 



15 Would you recommend this qualification to others? 

□ Yes 

□ No 

Please explain your answer 



16 About the assessment process 

How are you assessed? Please tick all that apply 

□ Written assignment 

□ Key Skills test 

□ Observation by assessor 

□ Witness testimony 

□ Spoken questions/interview 

□ Internal exam 

□ External exam 

□ Online tests 

□ Presentations 

□ Group work 

□ Simulation/mock situation 

□ Practical test 

□ Project work 

□ Video/audio recording 

□ Other 
Please state 



17 Did anyone explain to you the way(s) in which you were going to be 
assessed? 

□ Yes 

□ No 

If yes, who explained them to you? 

18 How often are you formally assessed? 

19 Which of the following best describes your view of the amount 
of support available to you in the run-up to being assessed? 

□ There is lots of support 

□ There is adequate support 

□ There is not enough support 

20 Which of the following best describes the quality of the support 
available to you in the run-up to being assessed? 

□ Support is of high quality 

□ Support is of satisfactory quality 

□ Support is of poor quality 

21 We are also interested in which assessment methods you prefer. 
Please rank the top 3 in order of preference with 1 = first 
preference, 2 = second preference, 3 = third preference 

Written assignment 

Key skills test 

Observation by assessor 

Witness testimony 

Spoken questions/interview 

Internal exam 

External exam 

Online tests 

Presentations 

Group work 

Simulation/mock situation 

Practical test 

Project work 

Video/audio recording 

Other 

Please state 



Appendix 3 



LSRC research report 



page 102/103 



About your ideas for improving the way you are assessed 

What has been most positive about the way you have been 
assessed for this qualification? 

1 

2 

3 



What has been most negative about the way you have been 
assessed for this qualification? 

1 

2 

3 



Thank you for your time. The sponsors for this research 
are City and Guilds and the Learning and Skills Research 
Centre and they will take your views into consideration 
when reviewing their awards. Please include your details 
if you want to be entered for the prize draw and return 
in the prepaid envelope provided. 

Name 



Contact details 
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Summary of key elements of questionnaire data 



Table 6 

Previous qualifications 
Missing responses: 35 



Table 7 

How are you assessed? 
(Tick all that apply) 



All students 

260 completed questionnaires returned from 890 distributed = 
34% return. 

Qualification N Valid percentage 



Degree 


2 


0.8% 


A Levels 


5 


2.2% 


NVQ 3 


1 


0.4% 


GNVQ 


8 


3.5% 


High School Diploma 


1 


0.4% 


BTEC 


2 


0.8% 


GCSE 5 or more at level C or above 


95 


42.2% 


GCSE less than 5 at level C or above 


82 


36.4% 


GCSEs none at level C or above 


9 


4.0% 


NVQ 1 or Progression Award 


7 


3.1% 


None 


13 


5.8% 


Total 


225 


100.0% 




N 


Valid percentage 


Written assignment 


224 


86% 


Observation by assessor 


138 


53% 


Spoken questions/interview 


87 


34% 


External exam 


112 


43% 


Presentations 


137 


53% 


Simulation/mock situation 


69 


27% 


Project work 


106 


41% 


Key skills test 


133 


51% 


Witness testimony 


62 


24% 


Internal exam 


100 


39% 


Online tests 


80 


31% 


Group work 


112 


43% 


Practical test 


95 


37% 


Video/audio recording 


14 


5% 
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Table 8 

How would you prefer to 
be assessed? 

(List top three in rank 
order ie 1 = highest 
ranking) 



a 

Analysed by mean score 

Assessment method 


N 


Mean score 


1 Written assignments 


133 


1.53 


2 Online tests 


44 


1.77 


3 Practical tests 


80 


1.86 


4 Observation by assessor 


59 


1.95 


5 External exam 


52 


1.98 


6 Project work 


73 


2.21 



b 

Analysed by numbers of students who chose the 
type of assessment as one of their top three 



Type of assessment N 

1 Written assignments 133 

2 Practical tests 80 

3 Project work 73 

4 Observation by assessor 59 

5 External exam 52 

6 Group work 47 



NB: 

These figures need to be linked to experience of assessment 
methods (previous question) and possible over-representation of 
sub-samples in the total. For example, 44 chose online tests as 
one of their top three methods of assessment but only 80 report 
having experience of it, virtually all of whom will be MVE 
apprentices who prefer online testing over paper and pencil 
testing; hence it is highly ranked in (a), but not (b). 



Table 9 




N = 


N = choosing 




Percentage of students 


Assessment method 


experience of 


1 of preferred top 3 


Percentage 


choosing form of 
assessment as 
preferred top three in 
relation to experience 


Written assignment 


224 


133 


59.% 


Observation by assessor 


138 


59 


43% 



Spoken questions/interview 


87 


33 


38% 


External exam 


112 


52 


47% 


Presentations 


137 


44 


32% 


Simulation/mock situation 


69 


18 


26% 


Project work 


106 


73 


69% 


Key skills test 


133 


11 


8% 


Witness testimony 


62 


2 


3% 


Internal exam 


100 


28 


28% 


Online tests 


80 


44 


55% 


Group work 


112 


47 


42% 


Practical test 


95 


80 


84% 


Video/audio recording 


14 


1 


7% 



This shows that the most popular forms of assessment by those 
who have experience of them are: 

1 Practical test 

2 Project work 

3 Written assignment 

4 Online tests 

5 External exam 

6 Observation by assessor 



Table 10 


Type of assessment 


N 


Valid percentage 


Are you intending 
to take further 
qualifications? 
Missing responses: 6 


Yes 


152 


59.8% 


No 


32 


12.6% 




Don't know 


70 


27.6% 



Total 



254 



100 . 0 % 
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Results by sub-groups 

NVQ Level 2 and 3, Modern Apprenticeships, 

Progression Award (Care and Motor Vehicle) 

102 completed questionnaires returned out of 460 distributed to 
this group of learners. 



Table 11 

Previous qualifications 
Missing responses: 23 



Table 12 

How are you assessed? 
(Tick all that apply) 



Qualification 


N 


Valid percentage 


A-levels 


3 


3.8% 


GNVQ 


7 


8.9% 


GCSE 5 or more at level C or above 


18 


22.8% 


GCSE less than 5 at level C or above 


26 


32.9% 


GCSEs none at level C or above 


9 


11.4% 


NVQ 1 or Progression Award 1 


7 


8.9% 


None 


9 


11.4% 


Total 


79 


100.0% 


How assessed 


N 


Valid percentage 


Written assignment 


75 


73% 


Observation by assessor 


86 


84% 


Spoken questions/interview 


53 


52% 


External exam 


46 


45% 


Presentations 


29 


28% 


Simulation/mock situation 


27 


26% 


Project work 


14 


14% 


Key skills test 


76 


74% 


Witness testimony 


56 


55% 


Internal exam 


32 


31% 


Online tests 


76 


74% 


Group work 


31 


36% 


Practical test 


30 


35% 



Video/audio recording 



3 



3% 



Table 13 

How would you prefer to 
be assessed? 



(list top three in rank 
order ie 1 = highest 
ranking) 



a 

Analysed by mean score 

Type of assessment 


N 


Mean score 


1 Written assignment 


24 


1.75 


2 Online tests 


42 


1.76 


3 Practical tests 


35 


1.80 


4 Observation by assessor 


41 


2.00 


5 External exam 


9 


2.00 


6 Internal exam 


5 


2.00 



b 

Analysed by numbers of students who chose the 
type of assessment as one of their top three 




Type of assessment 


N 


1 Online test 


42 


2 Observation by assessor 


41 


3 Practical tests 


35 


4 Written assignment 


24 


5 Spoken questions/interview 


20 


6 Group work 


12 



The above results are rather counter-intuitive, especially the 
top ranking in (a) of ‘written assignments’ and (b) online tests. 
Most candidates will encounter ‘written assignments’ either 
in college-based study or in writing reports on patients/clients 
(care plans etc). However, the preference for ‘written assignments’ 
in (a) may be skewed by Social Care NVQ candidates also 
interpreting their completion of portfolios as evidence of ‘written 
assignments’; the preference for online testing in (b) derives 
from MVE apprentices who prefer online multiple-choice tests 
over paper-and pencil multiple-choice tests. It may also be the 
case that some respondents identify methods of ‘assessment’ 
with formal classroom study and/or tests, and perhaps don’t 
even know that they are being formally assessed by, for example, 
routine observation in the workplace. 



N 



Table 14 

How often do you see 
your assessor? 
(Organised by 
contrasting categories) 



How often 

Once a month or more 



Every three months or less 



30 



10 



Valid percentage 

48% 



15% 



There was a low response to this question, with only 63 
responses. The remaining 23 who did respond reported that they 
saw their assessor every 6 weeks. 



Table 15 

For how long? 

(Again, organised by 
contrasting categories) 

77 students responded 
to this question 



How long N Valid percentage 

One hour or less 47 61% 



Two hours or more 19 28% 



NB: 



■ 4 students reported that they saw their assessor every 

3 months or less and for one hour or less (one for about an hour 
every 6 months). 

■ 1 student reported that he saw his assessor once a week for 
2-3 hours. 

■ The most reported frequency and duration of visit was every 
4-6 weeks for 1-2 hours (25 students). 



Table 16 

Are you intending to 
take further 
qualifications? 
Missing responses: 2 



How long 


N 


Valid percentage 


Yes 


33 


33% 


No 


26 


26% 


Don’t know 


41 


41% 



Total 



100 



100 % 



A-level (Business Studies and Sport) 

33 completed questionnaires returned out of 120 distributed 
to this group of learners. 



Table 17 




N 


Valid percentage 


Previous qualifications 


GCSE 5 or more at level C or above 


32 


97% 




GCSE less than 5 at level C or above 


1 


3% 




Total 


33 


100% 


Table 18 


How assessed 


N 


Valid percentage 


How are you assessed? 
(Tick all that apply) 


Written assignment 


27 


82% 



Observation by assessor 


2 


6% 


Spoken questions/interview 


2 


6% 


External exam 


25 


77% 


Presentations 


5 


15% 


Simulation/mock situation 


6 


18% 


Project work 


16 


48% 


Key skills test 


2 


6% 


Witness testimony 


0 


0% 


Internal exam 


8 


24% 


Online tests 


1 


3% 


Group work 


5 


15% 


Practical test 


15 


45% 



Video/audio recording 



4 



12 % 



Table 19 

How would you prefer to 
be assessed? 

(List top three in rank 
order ie 1 = highest 
ranking) 
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a 

Analysed by mean score of rankings 

Type of assessment 


N 


Mean score 


1 Practical test 


16 


1.56 


2 External exam 


20 


1.65 


3 Simulation/mock situation 


3 


2.00 


4 Presentations 


2 


2.00 


5 Written assignment 


21 


2.10 


6 Project work 


22 


2.27 



b 

Analysed by numbers of students who chose the 
type of assessment as one of their top three 




Type of assessment 


N 


1 Project work 


22 


2 Written assignment 


21 


3 External exam 


20 


4 Practical test 


16 


5 Group work 


5 


6 Internal exam 


4 



Table 20 

Are you intending to 
take further 
qualifications? 

Missing responses: 1 



How long 


N 


Valid percentage 


Yes 


20 


62.5% 


No 


2 


6.3% 


Don’t know 


10 


31.3% 



Total 



32 



100 . 0 % 



BTEC/AVCE/BND (Sport and Business Studies) 

38 completed questionnaires returned out of 130 distributed to 
these learners. 



Table 21 

Previous qualifications 
Missing responses: 3 



Table 22 

How are you assessed? 



Qualification 


N 


Valid percentage 


Degree 


2 


5.7% 


ISIVQ 3 


1 


2.9% 


GCSE 5 or more at level C or above 


21 


60.0% 


GCSE less than 5 at level C or above 


9 


25.7% 


None 


2 


5.7% 


Total 


35 


100.0% 


How assessed 


N 


Valid percentage 


Written assignment 


36 


95% 


Observation by assessor 


21 


55% 


Spoken questions/interview 


11 


29% 


External exam 


15 


39% 


Presentations 


30 


79% 


Simulation/mock situation 


13 


34% 


Project work 


22 


58% 


Key skills test 


23 


60% 


Witness testimony 


3 


8% 


Internal exam 


5 


12% 


Online tests 


1 


3% 


Group work 


14 


37% 


Practical test 


14 


37% 



Video/audio recording 



3 



8 % 



Table 23 

How would you prefer to 
be assessed? 

(List top three in rank 
order ie 1 = highest 
ranking) 



Table 24 

Are you intending to 
take further 
qualifications? 

Missing responses: 2 
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a 

Analysed by mean score of rankings 

Type of assessment 


N 


Mean score 


1 Written assignment 


19 


1.16 


2 Practical test 


10 


1.60 


3 Observation by assessor 


9 


1.78 


4 External exam 


9 


2.11 


5 Spoken questions/interview 


4 


2.25 


6 Simulation/mock situation 


8 


2.38 



b 

Analysed by numbers of students who chose the 
type of assessment as one of their top three 




Type of assessment 


N 


1 Written assignment 


19 


2 Presentations 


13 


3 Project work 


11 


4 Practical test 


10 


5 Observation by assessor 


9 


6 External exam 


9 



How long 


N 


Valid percentage 


Yes 


17 


47.2% 


No 


4 


11.1% 


Don’t know 


15 


41.7% 



Total 



36 



100 . 0 % 



Access students 

87 completed questionnaires were returned out of 180 



Table 25 


Qualification 


N 


Valid percentage 


Previous qualifications 
Missing responses: 9 


A-levels 


2 


2.6% 




Advanced GNVQ 


1 


1.3% 




High School Diploma 


1 


1.3% 




BTEC 


2 


2.6% 




GCSEs 5 or more at level C or above 


24 


30.7% 




GCSEs less than 5 at level C or above 


46 


59.0% 




None 


2 


2.6% 




Total 


78 


100.0% 


Table 26 


How assessed 


N 


Valid percentage 


How are you assessed? 
(Tick all that apply) 


Written assignment 


86 


100% 



Observation by assessor 


29 


33% 


Spoken questions/interview 


21 


24% 


External exam 


26 


30% 


Presentations 


73 


84% 


Simulation/mock situation 


23 


26% 


Project work 


54 


62% 


Key skills test 


32 


37% 


Witness testimony 


3 


3% 


Internal exam 


55 


63% 


Online tests 


2 


2% 


Group work 


62 


71% 


Practical test 


36 


41% 


Video/audio recording 


4 


4% 



Table 27 

How would you prefer to 
be assessed? 

(List top three in rank 
order ie 1 = highest 
ranking) 



Table 28 

Are you intending to 
take further 
qualifications? 

Missing responses: 1 
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a 

Analysed by mean score 

Assessment method 


N 


Mean score 


1 Written assignment 


69 


1.38 


2 Observation by assessor 


8 


1.88 


3 Project work 


34 


2.00 


4 Spoken questions/interview 


9 


2.11 


5 Internal exam 


17 


2.35 


6= External exam 


14 


2.36 


6= Presentations 


28 


2.36 


6= Group work 


25 


2.36 



b 

Analysed by numbers of students who chose the 
type of assessment as one of their top three 


Type of assessment 




N 


1 Written assignment 




69 


2 Project work 




34 


3 Presentations 




28 


4 Group work 




25 


5 Practical test 




19 


6 Internal exam 




17 


How long 


N 


Valid percentage 



Yes 


82 


95.3% 


Don't know 


4 


4.7% 


Total 


86 


100.0% 



Which course/qualification? 

All students were intending to enrol at university, most to do 
a degree (includingteacher training and nursing) and some 
to do a diploma in nursing. 



Further information 



For further information about the 
issues discussed in this publication 
please contact: 

Information Services 

Learning and Skills Research Centre 

Learning and Skills Development Agency 

Regent Arcade House 

19-25 Argyll Street 

London W1F7LS 

Tel 020 7297 9144 
Fax 020 7297 9242 
enquiries@LSDA.org.uk 

This Learning and Skills Development 
Agency publication results from 
research commissioned by the 
Learning and Skills Council. 





